Unable to remove cluster

Rancher 2.x
1

I deleted nodes inside the cluster which were provisioned using Rancher GUI (not imported). After that, I tried to delete the cluster but I was stuck with this message:

Rpc error: code = Unavailable desc = all SubConns are in TransientFailure; Removing host [10.45.20.12] from node lists; rpc error: code = Unavailable desc = all SubConns are in TransientFailure; waiting on cluster-provisioner-controller

I tried connecting to the rancher server kubernetes cluster and tried removing the namespaces associated with that cluster but still no luck.

project-8286p Terminating 180d

Did anyone experienced this? What was the fix?

Thanks…

2

It depends a bit on what you did exactly, how many nodes with what roles were in the cluster and what nodes did you delete? Regarding the stuck in terminating, see https://github.com/rancher/rancher/issues/14715#issuecomment-407900343 and https://rancher.com/docs/rancher/v2.x/en/admin-settings/removing-rancher/user-cluster-nodes/

3

Hello. I deleted all 3 nodes. All of them are etcd, worker, and controlplanes.

4

I was able to resolve it by pointing my context to the rancher server cluster end doing a “kubectl edit cluster” and replacing that cluster’s finalizers into “finalizers: []”

2 Likes
5

I tried that but i get a

{ “data”: “404 page not found\n” }

6

Having the same issue since Rancher 2.6

Can’t delete the cluster, nodes are already cleaned and removed. When trying to remove the finalizer, getting the same 404 error.

Thanks

1 Like
7

I am having this same issue with a digital cluster

8

I have the same issue. I opened up case on Github but there is no response from the team [2.6] Unable to remove cluster · Issue #34650 · rancher/rancher · GitHub

1 Like
9

In my case, Rancher was not deleting a cluster in broken etcd state, for days.
What helped was removing finalizers lines from the cluster objects using the following commands:

kubectl edit cluster …
kubectl edit clusters.management …

There are also other CRDs, but I only edited these two.

10

It does not work.
I tried to create oke cluster failed, then I could not remove it.
the version is 2.6

11

@ckwadi thank you, you’re reponse put me on the right track to delete a cluster that was stuck.

12

RKE1

kubectl edit clusters.rancher.cattle.io/<cluster_name>

RKE2

kubectl edit clusters.management.cattle.io/<cluster_name>

For both set finalizers: to finalizers: []

Using the Rancher GUI worked for RKE1 but gave { “data”: “404 page not found\n” } for RKE2

1 Like