I deleted nodes inside the cluster which were provisioned using Rancher GUI (not imported). After that, I tried to delete the cluster but I was stuck with this message:
Rpc error: code = Unavailable desc = all SubConns are in TransientFailure; Removing host [10.45.20.12] from node lists; rpc error: code = Unavailable desc = all SubConns are in TransientFailure; waiting on cluster-provisioner-controller
I tried connecting to the rancher server kubernetes cluster and tried removing the namespaces associated with that cluster but still no luck.
I was able to resolve it by pointing my context to the rancher server cluster end doing a “kubectl edit cluster” and replacing that cluster’s finalizers into “finalizers: []”
In my case, Rancher was not deleting a cluster in broken etcd state, for days.
What helped was removing finalizers lines from the cluster objects using the following commands: