I really like the rancher feature “run this container on every container” but it doesnt work with upgrading services or with using docker-compose files. When I create a stack using docker-compose the edit feature only lets me select scale 1+, there is no options for “one on every host”. Looking at the rancher compose files for services with such scaling, there is no scaling option in the rancher compose at all. So I tried creating services with a rancher-compose, and no scaling key, but that didnt work.
I have tried to add scheduling rules for the service so that they go across hosts, but this doesnt work well with sidekicks. I have to add a rule that they “should not run on host with service {my service name}” but this becomes hard to maintain with upgrades as the service name is changing every upgrade.
My main goal is to have high availability for my containers, so I would be happy with any of the following options.
- Containers chose to run on other hosts before stacking up on a single host. In other words, a single service should always span multiple hosts (if allowed by the scheduling rules) BEFORE putting multiple containers of the same service on the same host. Right now, everything tries to go to the initial host, creating a lot of imbalance.
- I could use “this service” in the scheduling rules. Then I could do a scheduling rule, “should not run on host with this service” and its easier to maintain then having the “hard-coded” service name.
- Allow me to switch to “one container per host” after creating a service.
Also as a slightly different issue, sidekicks should run on the same host as the main. So if I have 3 containers for my main service spread across 3 hosts, all three sidekicks are ran on host 1, there should be a 1-1 mapping per hosts. I cant change scheduling rules because a sidekicks inherits scaling rules from the main service.
Any ideas of how I can have HA services across hosts?