Rancher HA installation with external L7 load balancer

Rancher
1

I have followed instruction: https://rancher.com/docs/rancher/v2.x/en/installation/ha/rke-add-on/layer-7-lb/
to install Rancher (two nodes on VM’s) behind the Nginx LB on VM host. My Nginx config:

upstream rancher {
        server 192.168.101.11:80;
        server 192.168.101.12:80;
}

map $http_upgrade $connection_upgrade {
        default Upgrade;
        '' close;
}

server {
        listen 80;
        server_name rancher.domain.net;
        return 301 https://$server_name$request_uri;
        }

server {
        listen *:443 ssl http2;
        server_name rancher.domain.net;

        # SSL Configuration
        ssl_certificate /etc/letsencrypt/live/rancher.domain.net/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/rancher.domain.net/privkey.pem;
        ssl_trusted_certificate /etc/letsencrypt/live/rancher.domain.net/chain.pem;
        ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-    SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
        ssl_protocols TLSv1.2;
        ssl_session_timeout 1d;
        ssl_session_cache shared:SSL:50m;
        ssl_session_tickets off;
        ssl_prefer_server_ciphers on;
        add_header Strict-Transport-Security "max-age=63072000" always;
        ssl_stapling on;
        ssl_stapling_verify on;
        resolver 1.1.1.1;

        proxy_redirect off;
        client_max_body_size 100M;
        proxy_read_timeout 3000s;
        proxy_send_timeout 3000s;

        location / {
                allow 192.168.101.0/24;
                allow 1.2.3.4;
                allow 3.4.5.6;
                deny all;
                proxy_set_header Host $host;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header X-Forwarded-Port $server_port;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_pass http://rancher;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection $connection_upgrade;
                proxy_read_timeout 900s;
                proxy_buffering off;
        }
}

Unfortunately I’m getting error comming from the Rancher:
504 Gateway Time-out
openresty/1.15.8.1

Kubectl status:

$ kubectl get pods --all-namespaces
NAMESPACE       NAME                                      READY   STATUS      RESTARTS   AGE
cattle-system   cattle-7d7cfdcf4d-9w88q                   1/1     Running     0          12m
ingress-nginx   default-http-backend-5bcc9fd598-fnt4n     1/1     Running     0          13m
ingress-nginx   nginx-ingress-controller-6b899            1/1     Running     0          13m
ingress-nginx   nginx-ingress-controller-95t8p            1/1     Running     0          13m
kube-system     canal-9p56s                               2/2     Running     0          14m
kube-system     canal-n68fg                               2/2     Running     0          14m
kube-system     coredns-799dffd9c4-2x52p                  1/1     Running     0          14m
kube-system     coredns-autoscaler-84766fbb4-r2km9        1/1     Running     0          14m
kube-system     metrics-server-59c6fd6767-f2gd7           1/1     Running     0          13m
kube-system     rke-coredns-addon-deploy-job-x4gfd        0/1     Completed   0          14m
kube-system     rke-ingress-controller-deploy-job-bvwnk   0/1     Completed   0          13m
kube-system     rke-metrics-addon-deploy-job-ksxjr        0/1     Completed   0          13m
kube-system     rke-network-plugin-deploy-job-wghhq       0/1     Completed   0          14m
kube-system     rke-user-addon-deploy-job-mtdlw           0/1     Completed   0          13m

However cattle pod has been deployed to only one node (192.168.101.12)
So, I have made changes to my nginx configuration:

upstream rancher {
        server 192.168.101.12:443;
}

and

proxy_pass https://rancher;

That worked. Why the only one instance of Rancher has been deployed?

2

the rancher doc you linked is only for an older Rancher deployment method (before 2.0.8)

The current docs state layer 4 LB in front of nodes, and here is an example nginx config

https://rancher.com/docs/rancher/v2.x/en/installation/ha/create-nodes-lb/nginx/

3

With stream module I can’t use the same Nginx instance as a reverse proxy for other services except running it on the different port. Which is not something I was looking for. However, thanks for advise.