I have tried almost everything i can think of. I have a post here:
I believe my ceerts expired and I tried to do an ./rke cert rotate which resulted in a failure to find the old certs. I tried copying everything from /etc/kubernetes/ssl to /etc/kubernetes/.tmp per some documentation I found but to no avail.
``` rke cert rotate INFO Running RKE version: v1.2.6 INFO Initiating Kubernetes cluster INFO Rotating Kubernetes cluster certificates FATA Failed to rotate certificates: can't find old certificates ```
I also tried to remove my cluster.rkestate and delete my certs and tried to redeploy, but I am still met with timeout and TLS errors when trying to verify the cluster.
kubectl get cs Warning: v1 ComponentStatus is deprecated in v1.19+ NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-2 Unhealthy Get "https://18.104.22.168:2379/health": read tcp 22.214.171.124:38402->126.96.36.199:2379: read: connection reset by peer etcd-1 Unhealthy Get "https://188.8.131.52:2379/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) etcd-0 Unhealthy Get "https://184.108.40.206:2379/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I have even tried to rebuild a node, copy my ETCD backup and restore using RKE. This appeared to have worked, but all my my pods are stuck in a “pending” state due to have no “schedulable” nodes.
I also tried to take down my cluster to a single master node to reduce variables and tried to deploy using the existing cluster here. I am completely out of ideas and it might be time to rebuild. Wanted to reach out to see if anyone had any ideas. Thank you!