I’m trying to use Rancher to solve a very basic problem at the moment. I already have AWS autoscale groups that need specific stacks pushed to them. Ideally, I’d like to be able to tag a service group, say type=staging and also tag my servers when they autoscale through cloudformation with type=staging so that configured services know which nodes they are supposed to run on.
I understand how to set labels on the nodes when they boot up:
sudo docker run -e CATTLE_HOST_LABELS='type=staging' -d --privileged -v \
/var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher \
rancher/agent:v1.2.1 https://try-api.rancher.com/v1/scripts/redacted:1483142400000:redacted\n"
However, it’s not obvious to me how I can set this label on my service. Any help would be appreciated, or if I’m going about it the wrong way, some architecture advice would be great.
Labels are a property of services that you can set in the UI or compose, which directly corresponds to labeled on native docker containers.
You don’t need to label both sides though; you can either:
- Label hosts and put in scheduling rules that they must run on a host with label X
Or
- Label containers and put “required container labels” on the host so that the host will only run containers with one or more matching labels (Rancher 1.5+).
1 Like
Thanks for the reply. Here’s what I did (just so other users have a concrete example).
In my docker compose, I set a label on the service of type: staging
version: '2'
services:
web:
image: repo/image:latest
entrypoint:
- pm2-docker
- /home/code/process.yml
- --only
- web
ports:
- 80:3080/tcp
- 443:3443/tcp
labels:
type: staging
As my nodes spin up, they acquire the tag type=label from a UserData (startup) script:
sudo docker run -e CATTLE_HOST_LABELS=\"type=staging\" -d --privileged -v /var/run
/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.1 https://try-api.rancher.com/v1/scripts/<redacted>:<redacted>:<redacted>
I just found an inconsistency in how the label rules are followed:
Here is my node (you can see the label autoscale=staging):
Here is my services config (see type=staging):
So, seems to me that those services should not deploy to those nodes because the labels don’t match. Anyone care to comment on this and explain how I would limit to only properly labelled nodes?
You’re just labeling the hosts and services, but there is no particular relationship between those. That’s just two independent pieces of data, nothing is acting on it.
As I said above you need either to add a scheduling rule to make the service require a host with a certain label or add the service label to the list of labels the host(s) require. In either case you only need to label one side or the other, not both.
@vincent
Per your instructions:
Or
Label containers and put “required container labels” on the host so that the host will only run containers with one or more matching labels (Rancher 1.5+).
I labelled my containers via my docker-compose.yml file
version: '2'
services:
web:
image: repo/image:latest
entrypoint:
- pm2-docker
- /home/code/process.yml
- --only
- web
ports:
- 80:3080/tcp
- 443:3443/tcp
labels:
type: staging
And I labelled my hosts by adding it to the ranger/agent. If you don’t mind, could you provide an example or some steps that might assist me (and others) with a solution?
“required container labels” is not the same thing as “labels” on the host.
Thanks for the clarification. In my case, I need to be able to have the host come up with this as a requirement and without manual intervention. Is it possible to issue something when the host boots up to enforce this requirement?
sudo docker run -e CATTLE_HOST_LABELS='autoscale=docker' -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.1 https://try-api.rancher.com/v1/scripts/<redacted>:<redacted>:<redacted>
The “require container label” value is itself a host label, io.rancher.scheduler.require_any=io.rancher.container.system, type=staging.
So you can do that with CATTLE_HOST_LABELS, it just has to be urlencoded correctly, which the custom Add Host screen can give you:
Well, now things are better and worse… This is pretty bizarre, but now my host seems to be spawning tons of containers…
Here is how I called the forced labels:
sudo docker run -e CATTLE_HOST_LABELS='io.rancher.scheduler.require_any=io.rancher.container.system%2Cautoscale%3Ddocker' -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.1 https://try-api.rancher.com/v1/scripts/readacted\
Here is a screenshot of the Edit Host screen:
As you can probably see, I substituted type=staging for autoscale=docker to more accurately reflect my environment.
Did you delete the host before re-running the registration command? If you want to rerun registration just run the command again.
Deleting doesn’t do anything to the actual containers running on the host. If the host is then added back as a new host, all the existing containers are imported from it, and not tied to any previous service/stack. These are the ones that start with r-, as that is what the containers are actually named to docker.
I did delete the host before re-running the registration command. The other containers appear to be attempting to launch and then failing. When I issue:
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
adventhp/cavo latest 07e85d26e061 23 hours ago 2.513 GB
rancher/network-manager v0.5.3 0f224908d730 37 hours ago 241.6 MB
rancher/metadata v0.8.11 19b37bb3e242 2 weeks ago 251.5 MB
rancher/agent v1.2.1 e0fe21271bed 2 weeks ago 233.7 MB
rancher/scheduler v0.7.4 7a32d7571cad 3 weeks ago 241.9 MB
rancher/net v0.9.4 5ac4ae5d7fa4 3 weeks ago 264.4 MB
rancher/network-manager v0.4.8 45bdcd2b1944 3 weeks ago 241.6 MB
rancher/dns v0.14.1 4e37fc4150c2 3 weeks ago 239.9 MB
rancher/healthcheck v0.2.3 491349141109 8 weeks ago 383.3 MB
rancher/net holder bb516596ce5a 12 weeks ago 261.7 MB
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES
1ac7e1dd4e2c adventhp/cavo:latest "/.r/r pm2-docker /ho" 19 minutes ago Up 19 minutes
r-Docker-ocr-3-645e5370
e4f410959354 adventhp/cavo:latest "/.r/r pm2-docker /ho" 19 minutes ago Up 19 minutes
r-Docker-ocr-2-fe774390
021f5d5c33b9 rancher/network-manager:v0.5.3 "/rancher-entrypoint." 8 hours ago Up 8 hours
r-network-services-network-manager-8-07772297
4b0611ad40ce rancher/net:v0.9.4 "/rancher-entrypoint." 11 hours ago Up 11 hours
r-ipsec-ipsec-router-1-4388aa80
161843c46700 rancher/dns:v0.14.1 "/rancher-entrypoint." 11 hours ago Up 11 hours
r-network-services-metadata-dns-7-9bd280f6
5add41da7315 rancher/metadata:v0.8.11 "/rancher-entrypoint." 11 hours ago Up 11 hours
r-network-services-metadata-7-6e0c06d5
f8500326069d rancher/healthcheck:v0.2.3 "/.r/r /rancher-entry" 11 hours ago Up 11 hours
r-healthcheck-healthcheck-10-6f6a867a
8612210f61f1 rancher/scheduler:v0.7.4 "/.r/r /rancher-entry" 11 hours ago Up 11 hours
r-scheduler-scheduler-1-eaa2b727
856bbf6a1fc1 rancher/net:holder "/.r/r /rancher-entry" 11 hours ago Up 11 hours
r-ipsec-ipsec-1-2348495d
9128508d0758 rancher/net:v0.9.4 "/rancher-entrypoint." 11 hours ago Up 11 hours
r-ipsec-ipsec-cni-driver-1-d80c8e64
9519d74f2953 adventhp/cavo:latest "/.r/r pm2-docker /ho" 11 hours ago Up 11 hours
r-Docker-web-1-2846d616
2d6e07423e1f adventhp/cavo:latest "/.r/r pm2-docker /ho" 11 hours ago Up 11 hours
r-Docker-ocr-1-140b1800
374104677d31 adventhp/cavo:latest "/.r/r pm2-docker /ho" 11 hours ago Up 11 hours
r-Docker-edi-1-d91d172d
a5b3e25699e5 adventhp/cavo:latest "/.r/r pm2-docker /ho" 11 hours ago Up 11 hours
r-Docker-email-1-2eea4544
5f9a922840cc rancher/agent:v1.2.1 "/run.sh run" 11 hours ago Up 11 hours
I see a nice consolidated list of what is running on the host. Why does rancher show so many artifacts? Is there a way to reduce the clutter?
