Rancher plugin error stop the docker to come online after rebooting hosts


After rebooting the rancher hosts, it takes almost an hour for docker to become online since it is having trouble finding the volume rancher-secrets. Guess, plugins gets up only after the agent connect to rancher sever right? so is it some deadlock and docker comes up after a certain timeout?

Error as below

“Error restoring volume” driver=rancher-secrets error=“error while checking if volume “d0b8159dd26836cc4c009308b6bdc3d08d8b043432c34c07049742171e5fb6” exists in driver “rancher-secrets”: Error looking up volume plugin rancher-secrets: legacy plugin: plugin not found”


We have an open issue on this: https://github.com/rancher/rancher/issues/8984

Oh, so there is plan to fix the issue in 2.0 version only?

A docker “prune” command to clean unused environment may accelerate Docker starting process. It doesn’t fix the issue but limits waste of time…

I had a bad experience with docker prune before as it removed some additional layers which itself made the docker down forever, so I avoid using it if possible. Here, I tried removing by filtering unused volumes, out of 916 unused volumes(not why they create this much volumes), only 25 got removed. After I have tried a docker prune which doesnt reclaim any space means still more than 890 dangling volumes present.

#docker volume prune
WARNING! This will remove all volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Total reclaimed space: 0 B

#docker volume ls -q --filter “dangling=true” | wc -l

Seems for each volumes, docker is waiting for certain time and now it seems crossing 3 hrs for docker to be in active state