We’ve run into the problem that after a reboot on agents where we deployed rancher-nfs and services that depend on it, it takes a long time until docker becomes ready to use.
The problem is this:
Mär 21 12:47:52 hostname docker[4636]: time="2017-03-21T12:47:52.528929107+01:00" level=error msg="Handler for POST /v1.22/containers/02efe96f227f203559bc3e0518bb5c6399789dc12de1e91bce9b39c9c9efd07c/start returned error: get nexus: Error looking up volume plugin rancher-nfs: plugin not found"
Mär 21 12:47:53 hostname docker[4636]: time="2017-03-21T12:47:53.051485337+01:00" level=warning msg="Unable to locate plugin: rancher-nfs, retrying in 1s"
Mär 21 12:47:54 hostname docker[4636]: time="2017-03-21T12:47:54.051973390+01:00" level=warning msg="Unable to locate plugin: rancher-nfs, retrying in 2s"
Mär 21 12:47:56 hostname docker[4636]: time="2017-03-21T12:47:56.095197107+01:00" level=warning msg="Unable to locate plugin: rancher-nfs, retrying in 4s"
Mär 21 12:48:00 hostname docker[4636]: time="2017-03-21T12:48:00.101637440+01:00" level=warning msg="Unable to locate plugin: rancher-nfs, retrying in 8s"
docker does not start containers that require the rancher-nfs plugin as long as rancher-nfs isn’t available yet. Once docker has worked through its list of containers to restart far enough to reach rancher-nfs, everything is fine, but until then, it will try several times per container and it takes 16 seconds each time. This adds up.
This is Docker 1.10.3 on CentOS 7.2, Rancher 1.4.2. Has anyone found a solution for this? I’m aware this is a Docker issue rather than a Rancher issue, but this seems to be a situation others using rancher-nfs might have run into a well.