HAProxy as ELB for Rancher High Availability (HA)

Rancher 1.x
5

Hi all !

Me and some colleagues spent many time trying to install Rancher in HA mode last months !

Firts we started trying with CentOS 7 and met many difficulties, with devicemapper (we configured direct-lvm mode instead of loop-lvm for Docker). This configuration was not the good one. Perhaps, the kernel version was not appropriated for docker.

Finally we adopted Debian 8. We have a 3.14 kernel version. With this kernel we have some troubles with docker and devicemapper with direct-lvm mode. If we have to delete racher-ha-cattle we faced some kernel error like that:

unregister_netdevice: waiting for lo to become free. Usage count = -1

After that, we was not sure about how good was our installation. So we continue with Debian 8, but with the 3.16 kernel, and the Docker 1.10.3 ! This new installation uses AUFS. So we didn’t tried the devicemapper with direct-lvm.
In our installation we also faced a bug of the rancher HA installer script. We have to put our certificate in the script as it’s described here:

Loadbalancer

To ansewer to @moensch we use Nginx as a loadbalancer. And it’s work.

upstream rancher {
ip_hash;
server xxx.xxx.xxx.xxx;
server xxx.xxx.xxx.xxx;
server xxx.xxx.xxx.xxx;
}

server {

listen 80;
listen 443 ssl;
server_name <<sub domain>>.<<domain>>.<<tld>>;

ssl on;
ssl_certificate /etc/rancher/ssl/cert.pem;
ssl_certificate_key /etc/rancher/ssl/privkey.pem;
ssl_trusted_certificate /etc/rancher/ssl/chain.pem;

ssl_session_cache shared:SSL:20m;
ssl_session_timeout 10m;

ssl_prefer_server_ciphers       on;
ssl_protocols                   TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers                     ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;

add_header Strict-Transport-Security "max-age=31536000";

access_log /var/log/nginx/access.log;
error_log  /var/log/nginx/error.log;

location / {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Port $server_port;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://rancher;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    # This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close.
    proxy_read_timeout 900s;
}

}

I spent time, and time with Haproxy. I will investigate more on it. Now Rancher is working.

a new error

I face a last error when I try to add a host that will be manage by Rancher. It’s the same trouble as it’s described here:

Our certificate is generated with Let’s Encrypt.

Our stack:

  • Debian 8.5 (3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2+deb8u3 (2016-07-02) x86_64 GNU/Linux)
  • Docker 1.10.3
  • Rancher Server 1.1.2
  • Rancher Agent-instance v0.8.3
  • Rancher Agent v1.0.2

Cheers