I’ve been trying to setup a K3S cluster in HA with the following setup. (for the sake of simplicity fake IP’s)
1 proxy server - my-dns == 18.104.22.168
2 master servers - 22.214.171.124 & 126.96.36.199
2 agent servers - 188.8.131.52 & 184.108.40.206
I tried having both HAProxy and Nginx proxies and also tried using Cloudflare LB where I point ports 6443, 443 and 80 to the masters using round robin.
To run the k3s masters I ran: curl -sfL https://get.k3s.io | sh -s - server --datastore-endpoint=‘mysql://STRING_HERE’ --tls-san=‘my-dns’ --cluster-domain=‘my-dns’
I then follow the steps in the docs and eventually I get to the point to install rancher with Let’sEncrypt, so I ran the following:
helm install rancher rancher-latest/rancher \ --namespace cattle-system \ --set hostname=my-dns \ --set ingress.tls.source=letsEncrypt \ --set letsEncrypt.firstname.lastname@example.org
When I then run: kubectl -n cattle-system get ingress
I get the following:
NAME CLASS HOSTS ADDRESS PORTS AGE
rancher my-dns 80, 443 82s
cm-acme-http-solver-hljwn my-dns 220.127.116.11 80 60s
and it just stays like this for ever, the acme doesn’t seem to be working at all…
If I just skip the proxy and change my-dns to point to 18.104.22.168 it just works right away.
I’m trying to understand what could be wrong here, I feel like the address in the ingress should be the proxy and not the master01 ip but I don’t understand it much so I can’t be sure.
any how, I can’t seem to get this to work… any clues?
I’ve exhausted all my ideas and researched everything I could ideas would be welcome.