Cannot delete longhorn namespace

Hi all,
I have a k8s cluster internal setup by k3s.
I have installed longhorn by helm
helm install longhorn longhorn/longhorn --namespace longhorn -f values.yaml
Then I uninstall longhorn a few seconds later
helm -n longhorn uninstall longhorn
But something went wrong, I CTL+C, but then longhorn cannot be found with helm

helm -n longhorn list
NAME    NAMESPACE       REVISION        UPDATED STATUS  CHART   APP VERSION

However, the longhorn namespace is still there and I want to delete this namespace.
kubectl delete ns longhorn --grace-period=0 --force --cascade=true
hang for long time

kubectl get ns
NAME                STATUS        AGE
longhorn            Terminating   70m

kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get --ignore-not-found --show-kind -n longhorn
NAME                                  STATE      IMAGE                               REFCOUNT   BUILDDATE   AGE
engineimage.longhorn.io/ei-68f17757   deployed   longhornio/longhorn-engine:v1.5.3   0          25d         62m
NAME                              READY   ALLOWSCHEDULING   SCHEDULABLE   AGE
node.longhorn.io/k3s-hq-master1   True    true              True          62m
NAME                                                                                  VALUE                                         AGE
setting.longhorn.io/crd-api-version                                                   longhorn.io/v1beta2                           62m
setting.longhorn.io/current-longhorn-version                                          v1.5.3                                        62m
setting.longhorn.io/backup-target                                                                                                   62m
setting.longhorn.io/default-backing-image-manager-image                               longhornio/backing-image-manager:v1.5.3       62m
setting.longhorn.io/support-bundle-manager-image                                      longhornio/support-bundle-kit:v0.0.27         62m
setting.longhorn.io/backup-target-credential-secret                                                                                 62m
setting.longhorn.io/default-engine-image                                              longhornio/longhorn-engine:v1.5.3             62m
setting.longhorn.io/allow-recurring-job-while-volume-detached                         false                                         62m
setting.longhorn.io/default-instance-manager-image                                    longhornio/longhorn-instance-manager:v1.5.3   62m
setting.longhorn.io/create-default-disk-labeled-nodes                                 false                                         62m
setting.longhorn.io/default-data-path                                                 /var/lib/longhorn/                            62m
setting.longhorn.io/replica-soft-anti-affinity                                        false                                         62m
setting.longhorn.io/replica-auto-balance                                              disabled                                      62m
setting.longhorn.io/storage-over-provisioning-percentage                              100                                           62m
setting.longhorn.io/storage-minimal-available-percentage                              25                                            62m
setting.longhorn.io/storage-reserved-percentage-for-default-disk                      30                                            62m
setting.longhorn.io/upgrade-checker                                                   true                                          62m
setting.longhorn.io/allow-collecting-longhorn-usage-metrics                           true                                          62m
setting.longhorn.io/latest-longhorn-version                                                                                         62m
setting.longhorn.io/stable-longhorn-versions                                                                                        62m
setting.longhorn.io/default-replica-count                                             3                                             62m
setting.longhorn.io/default-data-locality                                             disabled                                      62m
setting.longhorn.io/default-longhorn-static-storage-class                             longhorn-static                               62m
setting.longhorn.io/backupstore-poll-interval                                         300                                           62m
setting.longhorn.io/taint-toleration                                                                                                62m
setting.longhorn.io/system-managed-components-node-selector                                                                         62m
setting.longhorn.io/auto-salvage                                                      true                                          62m
setting.longhorn.io/auto-delete-pod-when-volume-detached-unexpectedly                 true                                          62m
setting.longhorn.io/registry-secret                                                                                                 62m
setting.longhorn.io/disable-scheduling-on-cordoned-node                               true                                          62m
setting.longhorn.io/replica-zone-soft-anti-affinity                                   true                                          62m
setting.longhorn.io/node-down-pod-deletion-policy                                     do-nothing                                    62m
setting.longhorn.io/node-drain-policy                                                 block-if-contains-last-replica                62m
setting.longhorn.io/priority-class                                                                                                  62m
setting.longhorn.io/disable-revision-counter                                          false                                         62m
setting.longhorn.io/replica-replenishment-wait-interval                               600                                           62m
setting.longhorn.io/concurrent-replica-rebuild-per-node-limit                         5                                             62m
setting.longhorn.io/concurrent-volume-backup-restore-per-node-limit                   5                                             62m
setting.longhorn.io/system-managed-pods-image-pull-policy                             if-not-present                                62m
setting.longhorn.io/allow-volume-creation-with-degraded-availability                  true                                          62m
setting.longhorn.io/auto-cleanup-system-generated-snapshot                            true                                          62m
setting.longhorn.io/concurrent-automatic-engine-upgrade-per-node-limit                0                                             62m
setting.longhorn.io/backing-image-cleanup-wait-interval                               60                                            62m
setting.longhorn.io/backing-image-recovery-wait-interval                              300                                           62m
setting.longhorn.io/guaranteed-instance-manager-cpu                                   12                                            62m
setting.longhorn.io/kubernetes-cluster-autoscaler-enabled                             false                                         62m
setting.longhorn.io/orphan-auto-deletion                                              false                                         62m
setting.longhorn.io/storage-network                                                                                                 62m
setting.longhorn.io/failed-backup-ttl                                                 1440                                          62m
setting.longhorn.io/recurring-successful-jobs-history-limit                           1                                             62m
setting.longhorn.io/recurring-failed-jobs-history-limit                               1                                             62m
setting.longhorn.io/support-bundle-failed-history-limit                               1                                             62m
setting.longhorn.io/deleting-confirmation-flag                                        false                                         62m
setting.longhorn.io/engine-replica-timeout                                            8                                             62m
setting.longhorn.io/snapshot-data-integrity                                           fast-check                                    62m
setting.longhorn.io/snapshot-data-integrity-cronjob                                   0 0 */7 * *                                   62m
setting.longhorn.io/snapshot-data-integrity-immediate-check-after-snapshot-creation   false                                         62m
setting.longhorn.io/restore-volume-recurring-jobs                                     false                                         62m
setting.longhorn.io/remove-snapshots-during-filesystem-trim                           false                                         62m
setting.longhorn.io/fast-replica-rebuild-enabled                                      true                                          62m
setting.longhorn.io/replica-file-sync-http-client-timeout                             30                                            62m
setting.longhorn.io/backup-compression-method                                         lz4                                           62m
setting.longhorn.io/backup-concurrent-limit                                           2                                             62m
setting.longhorn.io/restore-concurrent-limit                                          2                                             62m
setting.longhorn.io/log-level                                                         Info                                          62m
setting.longhorn.io/v2-data-engine                                                    false                                         62m
setting.longhorn.io/v2-data-engine-hugepage-limit                                     1024                                          62m
setting.longhorn.io/offline-replica-rebuilding                                        enabled                                       62m
Warning: metallb.io v1beta1 AddressPool is deprecated, consider using IPAddressPool

Please give me some advice, thank you very much.

I found solution here : Longhorn Namespace Stuck Terminating - Delete Longhorn from Kubernetes Cluster
Patch all the CRDs and delete the finalizers. This can be done with this command: for crd in $(kubectl get crd -o name | grep longhorn); do kubectl patch $crd -p '{"metadata":{"finalizers":[]}}' --type=merge; done;

1 Like

Before you uninstall Longhorn, you should set the setting deleting-confirmation-flag to true.
If you delete the namespace directly, there might be some CRs that will never be deleted because the webhook server is deleted.

WHERE do we apply this flag?

@thezog Try kubectl -n longhorn-system edit settings.longhorn.io deleting-confirmation-flag

1 Like