Rancher HA Stack "Degraded"

I installed the Rancher V1.0.1 HA after installing two stack got the status of “degraded” the names of the containers are “rancher-compose-performer” and “go-machine-service” another problem that I realized that the browser when will the container can not access the shell of the container …
I’m new in if you need please logs tell me where to get that post here the topic.

thank you,

Fernando Felicissimo

Hi Fernando,

I am sorry to hear you are having trouble with HA. I have a few questions to better understand your setup.

Is your HA installation behind a load balancer? Are you accessing Rancher UI via port 80 or 443? 18080 is a diagnostic port and certain things will not work such as shell and log information on containers. We’ve left it open for troubleshooting the HA system stack, but it is not recommended to use.

How many hosts have you registered, and to what environment? Are you running your servers using a cloud provider? We’ve seen a lot of issues related to network security configuration as well.

I’m using external load balancer BigIP (F5), with respect to port use port 80, have 3 Hosts that are on my internal Data Center.
If you need logs to analyze my case … please tell me which will need and where to get them …
thank you,

Fernando Felicissimo

From one of your UI nodes, could you send me logs for the rancher-ha, rancher-ha-cattle, rancher-ha-zk containers? You may do this using docker ‘logs’ command:
docker logs rancher-ha

You mention 3 hosts, for a total of 6 correct? Or are you launching agents on the same hosts as your servers? Are these hosts visible on the INFRASTRUCTURE > HOSTS screen?

Hi James,

I have 3 physical servers that make up the cluster, the 3 hosts appears in infrastrture / hosts and networks of agents are properly started on all servers.
How can I send you the requested logs?




Hi, James

I made an attempt to make a reboot in 3 servers … After the reboot the rancher island is not more affordable …


Hi, James

Hi James,

Efetuei a 'docker -a ps "on the host realized that some container does not start more after the reboot … I took a screen print to show …

It looks like you’ve attempted to register the UI servers as agents. This is not recommended/supported. Please start new servers and register them to your cluster.

More info on this process here:

Hi James,

In cluster installation was generated rancher-ha.sh script that I downloaded and ran the other members of the cluster hosts, this is correct procedure?


Fernando Felicissimo

Correct, but not complete.

After you’ve done that, you need to add hosts to an environment that is not System-HA. System-HA should never be used for anything but the system. We expose it for debugging purposes.

Hello James,

So if correctly understood your explanation I run script “rancher-ha.sh” on all cluster member hosts in my case SERVER A SERVER B SERVER C they already will be automatically added to the “System HA” … after this will create an example environment “DevOps” and add the SERVER a SERVER B SERVER C and the environment “DevOps”, correct?

If I am correct I’m trying to add hosts “SERVER A SERVER B SERVER C” on the environment “DevOps” and the hosts do not appear in the environment “DevOps” created.

I’m putting down the host registry script on Rancher:

sudo docker run -d -e --privileged CA_FINGERPRINT = "3F: B4: B5: FB: BD: 91: 5E: 0D: C2: 81: A7: 73: FD: 41: C4: F0: E1: 31: 5E : 50 "-v -v /var/run/docker.sock:/var/run/docker.sock / var / lib / rancher / var / lib / rancher rancher / agent: v1.0.1http://phenix.magazineluiza.intranet/v1/scripts/5775434B167A817815F9:1461693600000:6QhhQ0hSnR0fzI3qwSDEaln960


Fernando Felicissimo

I’ll reiterate that it is not recommended to use Server A B C for anything but running rancher server, BUT there should be no problem doing so for development purposes.

Could you send me docker logs for the rancher agent? You may need to specify -e CATTLE_AGENT_IP=X.X.X.X for host registry if Rancher agent is picking the wrong IP address.

Is there any firewall between your servers, or wide open?

Hello James,

I did not understand the question not to be recommended “Host A, Host B, Host C” for the implementation of HA specifies the minimum setting is 3 servers.

The url “phenix.magazineluiza.intranet” is my load external balance (BIG IP - F5), then have to put the IP direct HOST?

The environment “System HA” I still have two containers in Management which is restarting … I ran the command “docker -f logs IDCONTAINER” shows me the following log:

time = “2016-04-26T19: 10: 01Z” level = fatal msg = “Unable to create event router” error = "Get http: //phenix.magazineluiza.intranet/v1: net / http: request canceled while waiting for connection "

Since you can see the UI, this works:
your ip address -> F5 -> server A|B|C

What is not working is:
server A|B|C -> F5

Your Rancher servers can’t communicate with the F5 load balancer. Please check your security configuration to ensure that they have access to port 80 on the F5.

I’m actually glad that you brought this to my attention, the HA documentation doesn’t reflect this requirement anywhere. We will get this info added.

Hello James,
I am enabled sniffer on the firewall to see if this having any HTTP / HTTPS lock as you said above.
thank you,

Great, I hope it helps.

I didn’t answer your other question about recommended servers. You are correct that 3 servers are recommended for Rancher HA. That number is in addition to servers running Rancher agents, 1 or more is required (for a total of 4+). You may reuse the 3 servers running Rancher HA, but it is not recommended for production.

Hi James,

What you’re telling me that recomedada architecture for production would be:

03 Hosts for Rancher HA;
02 hosts for cluster docker considering failover and load balance both containers as the Hoost;

For a total of 5 hosts would be that the idea?


Fernando Felicissimo

For development or proof of concept, you could use only 3 servers:

For production HA, you would use separate servers for agents:

Only thing that changes is you run rancher-agent on new servers. The Agent hosts are where your catalog items are deployed to. You are on the right track that you would load balance whatever is running on Agent hosts.