Why are there multiple rancher-agent instances on my host?

Rancher 1.x
1

I just started a clean install of rancher-server, and put agent on another host.

Output of docker ps (I remove my own containers):

# docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS              PORTS                                          NAMES
c56b59b53aa8        rancher/agent-instance:v0.3.1   "/etc/init.d/agent-in"   22 minutes ago      Up 22 minutes       0.0.0.0:500->500/udp, 0.0.0.0:4500->4500/udp   83a46b24-65c3-42d4-8124-8ead1a953d01
354cf94ce71a        rancher/agent:v0.7.11           "/run.sh run"            34 minutes ago      Up 34 minutes                                                      rancher-agent
```

Why are there multiple agents, both running and stopped?

Second, my rancher UI is attached. 

<img src="//cdck-file-uploads-global.s3.dualstack.us-west-2.amazonaws.com/business7/uploads/suse/original/1X/ca44fd543c61b20f89aae9df94ca5782f5c8fcef.png" width="282" height="360"> 

Is that related to the agents showing up in http://forums.rancher.com/t/rancher-agents-showing-up-in-docker-1-8/247
2

Yes, it’s related. Does your fresh install have Docker 1.8+? The next release will correct it for fresh installs.

3

Yes, 1.8.1.

So, why are there multiple agent containers? I thought it was just one?

4

Hmm, and now there are 4 of them. 1 of rancher/agent:v0.7.11 and 3 of rancher/agent-instance:v0.3.1. What is it doing?

5

rancher/agent is the host to server communication agent that is normally not shown in the UI.

rancher/agent-instance is the base image for the “Network Agent”, which runs the IPSec, DNS, and other services for the other containers on that host. It is also used for each individual “LB Agent”.

So without seeing a newer docker ps -a, you may have several balancer containers plus the network agent plus the rancher agent.

6

So, for any given host, I will have:

  • 1 rancher/agent which I ran to connect the host to the rancher server
  • 1 rancher/agent-instance which the rancher/agent started to run networking for the host, if I use rancher IPSec networking
  • 0, 1 or more rancher/agent-instance to do the load balancing

Is that correct?

7

Running ones, yes.

There may also be some containers that will be removed from upgrades, switching from :latest to a concrete version , or installing depending on exactly how the agent was started it can create intermediary containers.

The rancher/agent containers should be hidden from within Rancher UI/API, that is not happening in the current release due to the issue with Docker 1.8 label inheritance.

8

Understood, thank you. I got the part about the 1.8 label inheritance issue. At this point I was trying to understand the container structure on a host.

Can you post a link to the changes in container label inheritance? Like to understand it.

9

I know there is a docker issue for it somewhere but a quick minute of searching didn’t turn it up. Basically Rancher (ibuildthecloud) contributed label support into Docker and we (intentionally) made it so that if an image has labels they are automatically applied to containers that use it. In 1.8 Docker (intentionally) broke that into requiring a separate lookup of the image and the labels on it rather than automatically inheriting them.

So there’s a label on the rancher/agent image that say “I’m a system container”, and that’s no longer getting applied to the container where we are looking. The next Rancher release works around it by checking the labels on the image.

10

Got it. I can see why they would do that, although personally I would have just had 2 classes of labels on the container itself - image and container - so it doesn’t require a separate lookup.

11

Yup… Since it was released the other way it’s a breaking API change that got sort of glossed over. And I didn’t make the change but believe the general workaround is actually more like n+1 lookups now because you have to look at each layer.

12

So, is this lack of product management maturity on their part - “we cannot see how the current structure would be useful or changing it would hurt” - or just politics?

BTW, if this is a topic that should be taken offline, both Will and Dave G have my email. Feel free to respond privately.

13

Here’s the issue… or rather pull request, which was why I didn’t find it this morning: https://github.com/docker/docker/pull/15503

14

Heh, someone got rather annoyed there. :slight_smile: