Hello
Here is my problem:
In the rangcher webgui i see errors:
Reason: Unhealthy
Object: Pod kube-apiserver-rancher-n1
Message: "Readiness probe failed: Error from server (InternalError): an error on the server ("[+]ping ok\n[+]log ok\n[-]etcd failed: reason withheld\n ..."
When executing:
kubectl exec -n kube-system etcd-rancher-n1 -- etcdctl --endpoints=https://10.0.8.49:2379,https://10.0.8.33:2379,https://10.0.8.0:2379 endpoint health
I get:
{"level":"warn","ts":"2024-10-02T17:07:42.688873Z","logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0004ca1e0/10.0.8.33:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\""}
{"level":"warn","ts":"2024-10-02T17:07:42.688794Z","logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00041a000/10.0.8.49:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\""}
{"level":"warn","ts":"2024-10-02T17:07:42.691162Z","logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0004ca000/10.0.8.0:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\""}
https://10.0.8.33:2379 is unhealthy: failed to commit proposal: context deadline exceeded
https://10.0.8.49:2379 is unhealthy: failed to commit proposal: context deadline exceeded
https://10.0.8.0:2379 is unhealthy: failed to commit proposal: context deadline exceeded
Error: unhealthy cluster
My setup:
Rancher HA Cluster
Node0: rancher-lb (nginx loadbalancer, with self-signed cert on it)
Node1: rancher-n1
Node2: rancher-n2
Node3: rancher-n3
During rancher install i choose setup with rancher autogenerated self-signed cert.
So installed cert-manager, and then rancher with command:
helm install rancher rancher-stable/rancher --namespace cattle-system --set hostname=rancher.mydomain --set bootstrapPassword='mypassword'
How to fix it?