Multiple storage driver for one environment

Archive Rancher 1.x
1

Hello guys,

When I started my future environment I choosed the rancher-nfs driver to manage my share storage. However I’m unable to make it work due to issue on the NFS server side which doesn’t accept connexion from source port > 1024…

So now I’d like to try another approach by mounting the NFS directory on the rancher host and use locale storage driver (named) from the container side to point to the right local directory.

But I don’t know how to mix the actual rancher-nfs driver with a local volume create within docker (docker volume create) directly manage by rancher.

Is someone from the rancher team could point me to the right direction ?

Cheers,
Renaud

2

Hi @renaudManda,

Do you have access to your NFS server’s exports? If so, without investigating the root cause of why non-privileged ports are necessary in your setup (they shouldn’t be), you can specify the insecure flag to your folder export. For example, this entry could be placed in /etc/exports:

/data *(rw,no_subtree_check,no_root_squash,insecure)

Out of curiosity, I tried your workaround and it actually worked, but I wouldn’t recommend it. What I did was this:

  1. Mount NFS to local volume driver storage path mkdir -p /var/lib/docker/volumes/test && mount -t nfs 1.2.3.4:/data/test /var/lib/docker/volumes/test
  2. Create the ‘local’ storage driver docker volume create -d local test
  3. Mount the named volume into the container of your choice.

Hope this helps,
James

3

Hello @LLParse,

First of all thanks for your quick reply. Unfortunately I don’t have any access to the NFS server because it’s an external service provide by my ISP, and I asked them previously for the insecure flag but without success.

Thanks for your workaround I’ll try and I’ll give you some feedback for history. However from your point of view what can go wrong with this approach?

cheers,
Renaud

4

Hmm, too bad. It’s interesting that you are running into this issue at all - I presume you are using the latest version of the rancher-nfs storage driver and nfsv4… are there any other custom mount options being provided? I’ve been able to successfully create nfsv4 mounts using the latest driver, without setting insecure.

As for the workaround, there could possibly be problems if the folder isn’t mounted prior to creating the named volume and/or bind-mounting into the container. The timing there needs to be perfect, otherwise your container will write to the underlying filesystem, instead of the mounted filesystem. Server restarts might also present problems if your docker daemon starts prior to the mount being made (make sure to use fstab entries of course). Instead of using a named volume, you could also just bind-mount the host path, eliminating the local storage driver hackery.

James

5

Well the bad news is I can only use nfsv3 :frowning: it’s probably why I have to use the insecure flag… In all cases my ISP (OVH) confirm me the port issue.
Maybe if I could mount the rancher-nfs container bind directly to the host (bypassing the masquerading) it could work but I don’t know how to accomplish that.

I note that point for my future workflow. I think (but I haven’t try yet) to check the NFS status before starting the docker-engine. I should be able to accomplish that with the dependencies of systemd but I have to look further forward to check that point.

Thank you very much for your workaround and for the rest,
If you find a solution about mounting nfsv3 directory in my use case don’t hesitate to send me it :wink:

Cheers,
Renaud