Hi
I reviewed similar topics on this certificate subject but could not find a solution. I installed Rancher 2.5.1 using my own certs. The command I used:
helm install rancher rancher-latest/rancher --namespace cattle-system --set hostname=rancher-draco.asc-dev.io --set ingress.tls.source=secret --set privateCA=true --kubeconfig ./kube_config_cluster.yaml
The Rancher GUI came up and when I created a new cluster, it gives me an error:
[etcd] Failed to bring up Etcd Plane: etcd cluster is unhealthy: hosts [10.0.9.72] failed to report healthy. Check etcd container logs on each host for more information
Using the kubectl describe command, the etcd log is throwing:
tls: failed to verify client’s certificate: x509: certificate signed by unknown authority (possibly because of “crypto/rsa: verification error” while trying to verify candidate authority certificate “kube-ca”)", ServerName “”
I found a Rancher documentation at: https://rancher.com/docs/rancher/v2.x/en/installation/resources/encryption/custom-ca-root-certificate/
I attempted to follow ‘Kubernetes Install’ directions from this documentation but it did not solve the problem.
When I ran bash command on rancher pod, I found the error that it could not find ca-additional-volume, so I manually copied cacerts.pem and ca-additional.pem into /etc/rancher/ssl and the errors in the rancher pod went away, however etcd is still not happy.
Any help you can provide to resolve this would be greatly appreciated.
Ram Mahajan
Hallo,
same here using catalogs.
Ralf
Additional information:
openssl verify -CAfile ./cacerts.pem tls.crt
tls.crt: OK
v3/settings/cacerts is showing the correct CA cert as well.
Interesting, it seems like all the tutorials I’ve seen for Rancher in HA doesn’t show how to get past this SSL problem when running the Rancher Agents.
“x509: ECDSA verification failure” while trying to verify candidate authority certificate “dynamiclistener-ca”
You could modify deployment of rancher agent in a such way so that, you add custom CA to accepted certificates inside that pod - it’s a hack’ish solution though…
Let’s say you are using Let’s encrypt Staging certificates:
wget https://letsencrypt.org/certs/staging/letsencrypt-stg-int-r3.pem
wget https://letsencrypt.org/certs/staging/letsencrypt-stg-root-x1.pem
echo “Creating secret with let’s encrypt certs for rancher agent pod”
kubectl -n cattle-system create configmap ca-pemstore --from-file=letsencrypt-stg-root-x1.pem --from-file=letsencrypt-stg-int-r3.pem
echo “### patching rancher agent deployment…”
echo “### adding volume from configmap with let’s encrypt certs”
kubectl -n cattle-system get deployment cattle-cluster-agent -o json | jq ‘.spec.template.spec.volumes += [{“name”: “ca-pemstore”, configMap: {“name”: “ca-pemstore”} }]’ | kubectl replace -f -
sleep 1
echo “### Mounting volume with cert”
kubectl -n cattle-system get deployment cattle-cluster-agent -o json | jq ‘.spec.template.spec.containers[0].volumeMounts += [{“name”: “ca-pemstore”,“mountPath”: “/usr/local/share/ca-certificates/letsencrypt-stg-root-x1.crt”,“subPath”: “letsencrypt-stg-root-x1.pem”,“readOnly”: false}]’ | kubectl replace -f -
sleep 1
echo “### Adding postStart action to update system wide certificates”
kubectl -n cattle-system get deployment cattle-cluster-agent -o json | jq ‘.spec.template.spec.containers[0].lifecycle.postStart.exec.command = [ “/usr/sbin/update-ca-certificates”]’ | kubectl replace -f -
