We need to migrate the content of /var/lib/rancher to a secondary disk because of space problems.
In order to do that, we thought that:
- making a first rsync to the secondary disk
- stopping and disabling rancher-agent and rke2 server services
- rebooting the node
- rsync again
- clear the current content of
/var/lib/rancher - mount the secondary disk under
/var/lib/rancher - enable and start rancher-agent and rke2-server
would do the trick. Unfortunately not!
Unfortunately, the first rsync takes ages to complete.
Upon restart of the services, the pods keep bouncing and the node never finishes converging.
I then tried to follow the steps described here: Removing Kubernetes Components from Nodes | Rancher
The idea is to remove the node from the cluster. Clean the /var/lib/rancher and re-provision the node once the secondary disk is mounted.
As described, I’m going to Rancher UI and deletes the node from the cluster.
According to the documentation, the deletion process should trigger a “cleanup”.
It does not seem to be the case: the node automatically gets back in the cluster and the folders are not cleared.
Am I doing something wrong?
Should the script system-agent-uninstall.sh be called at some point?