Upgrade from 2.4.6 -> 2.5.1

Rancher 2.x
1

I upgraded my Rancher installation (standalone docker container) and there are two messages in the Rancher log popping up every second:

time="2020-10-09T18:00:31.837530296Z" level=info msg="Waiting for containerd startup: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2020-10-09T18:00:32.063809685Z" level=info msg="Waiting for master node local-node startup: nodes \"local-node\" not found"

Clicking on ‘local’ cluster opens page with ‘Graph information from Cluster currently not available’ and it has 0 nodes.
Other cluster (the one that was managed by Rancher before) seems to be OK and functional.

UPDATE: after VM restart I’m not seeing these 2 log messages, but there are these logs now:

[storage/driver] 2020/10/09 18:36:04 query: failed to query with labels: the server could not find the requested resource (get configmaps)
UPGRADE FAILED
Error: the server could not find the requested resource (get configmaps)
2020/10/09 18:36:04 [ERROR] AppController p-m9qr2/rancher-k3s-upgrader [helm-controller] failed with : failed to install app rancher-k3s-upgrader. Error: UPGRADE FAILED: the server could not find the
 requested resource (get configmaps)
E1009 18:36:20.846641      25 scheduler.go:638] error selecting node for pod: no nodes available to schedule pods

Local cluster state seems to be stuck in ‘Upgrading’.

Is there any way to bring it back to normal or is it normal?

1 Like
3

@vprocopan Can you share the output of docker info and share if you have mounted /var/lib/rancher to somewhere on the host and if so, what filesystem/kind of mount that is?

1 Like
5

@superseb Thanks for trying to help me! @Dmitry_Shultz Here is what i did: You can just create a new XFS filesystem on your new disk and mount it at /var/lib/docker . Then you do not need to make any changes to the Docker configuration, as that is the default directory for Docker volumes, images, containers, etc. It worked for me!