Issue with nfs-provisioner from Catalog

Hello,

I’m trying nfs-provisioner server from Rancher Catalog.
My pvc is bounded well, but I got issue to mount it when my pod is not deployed on the same host than nfs-provisioner pod.
That’s annoying because I have a three nodes cluster and I have selected one node for the nfs-provisioner (because of hostPath value).
When my pod asking for volume is on the same host, the pvc is well mounted and works.
But if my pod is deployed on another node of the cluster, I got a timeout error at the volume mount.

Here are errors messages:

Type Reason Message Last Seen
Warning FailedMount Unable to attach or mount volumes: unmounted volumes=[test], unattached volumes=[test default-token-rvpqt]: timed out waiting for the condition a few seconds ago
Warning FailedMount MountVolume.SetUp failed for volume “pvc-86ec5e14-23d5-4b16-ad0e-0596f1abb0f0” : mount failed: exit status 32 Mounting command: mount Mounting arguments: -t nfs -o noatime,vers=4.1 10.43.155.226:/export/pvc-86ec5e14-23d5-4b16-ad0e-0596f1abb0f0 /var/lib/kubelet/pods/0dea2d33-5b3c-40af-8b5e-f6b03d69cdd6/volumes/kubernetes.io~nfs/pvc-86ec5e14-23d5-4b16-ad0e-0596f1abb0f0 Output: mount.nfs: Connection timed out a minute ago
Warning FailedMount Unable to attach or mount volumes: unmounted volumes=[test], unattached volumes=[default-token-rvpqt test]: timed out waiting for the condition 7 minutes ago
Normal Scheduled Successfully assigned default/test-5948fd8fb9-x5fms to k8s-wrkr-pprd-02 Invalid date

Do you have any ideas of what could cause that issue ?
Do you know what “waiting for the condition” could mean ?

Thanks a lot!

Maybe you can try to log in (via SSH) on the other cluster/worker nodes… can you mount the nfs share manually (within the RancherOS)? Another option would be running the nfs-provisioner pod on every worker node (3 replica’s)… also better for resiliency.