Rancher-monitoring is stuck in uninstalling state

Hello everyone,

we wanted to remove rancher-monitoring using the helm tool.

The following error message occurs:

uninstall.go:124: [debug] uninstall: Failed to delete release: [unable to build kubernetes objects for delete: [resource mapping not found for name: "rancher-monitoring-alertmanager" namespace: "cattle-monitoring-system" from "": no matches for kind "Alertmanager" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-prometheus" namespace: "cattle-monitoring-system" from "": no matches for kind "Prometheus" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-alertmanager.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-config-reloaders" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-etcd" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-general.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-k8s.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-apiserver-availability.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-apiserver-burnrate.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-apiserver-histogram.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-apiserver-slos" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-prometheus-general.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-prometheus-node-recording.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-state-metrics" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubelet.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubernetes-apps" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubernetes-resources" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubernetes-storage" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubernetes-system-apiserver" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubernetes-system-controller-manager" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubernetes-system-kube-proxy" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubernetes-system-kubelet" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubernetes-system" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-node-exporter.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-node-exporter" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-node-network" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-node.rules" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-prometheus-operator" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-prometheus" namespace: "cattle-monitoring-system" from "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-grafana" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-state-metrics" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-prometheus-node-exporter" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-controller-manager" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-etcd" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-ingress-nginx" namespace: "kube-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-proxy" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kube-scheduler" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-alertmanager" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-coredns" namespace: "kube-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-apiserver" namespace: "default" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-kubelet" namespace: "kube-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-operator" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first, resource mapping not found for name: "rancher-monitoring-prometheus" namespace: "cattle-monitoring-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first]]

How exactly do you proceed in such a case? Can you reinstall the missing CRDs manually?

Thanks in advance for any advice.

We were able to fix the issue by removing all secrets related to Helm:

kubectl -n cattle-monitoring-system get secrets | grep -i helm | grep -v NAME | awk '{print $1}' | xargs kubectl -n cattle-monitoring-system delete secret

You can now delete all remaining resources manually or put the chart back into operation. We did the latter.