Mounting external NFS shares on RancherOS nodes

Archive RancherOS
1

Hi there,

I’d like to mount an NFS location onto my rancher hosts to provide external storage/backup/etc. I’ve seen suggestions on how to do this on CoreOS (a systemd/fleet unit to start statd, another to mount the volume). However, any manual mount commands I run on the rancher node as a test seem to get permutations on “file not found” and the lack of the standard nfs client tools (rpcinfo etc) in rancheros suggest I’m doing something that was really not meant to be.

I can mount the NFS share inside a docker container running with --cap-admin=SYS_ADMIN and the nfs-common packages installed, but I can’t work out how to make the mount visible to other containers (let alone the host) without just making every container an NFS client.

Any thoughts from anyone on how to do this well? Or should I take from my googling the message that, if you can’t do it on the host OS, don’t do it at all?

Thanks for your time,

-michael

2

Hi Michael,

you can use privileged containers to mount NFS filesystems and export them as docker volumes for use by other containers. This compose YML template example might give an idea:

nfs-volumes:
  image: your/nfs-client
  privileged: true
  net: host
  environment:
    NFS_MOUNT_01: stor.example.com:/misc/export1 /opt/data/datavol1
    NFS_MOUNT_02: stor.example.com:/misc/export2 /opt/data/datavol2
  volumes:
  - /opt/data/datavol1
  - /opt/data/datavol2

worker1:
  image: my/worker1
  volumes_from:
  - nfs-volumes

worker2:
  image: my/worker2
  volumes_from:
  - nfs-volumes

Here worker1 and worker2 don’t have to be NFS clients themselves: they just use the data volumes from nfs-volumes container, which is responsible for mounting NFS shares from NFS_MOUNT_* env vars.

2 Likes
3

Thanks for that! I will test tonight. I thought I had tried that but, on consideration, I think I had the volumes defined on the next-path-up - ie, on /opt/data, rather than /opt/data/datavol1 - which only showed the underlying /opt/data content through to the worker1/2 containers, not the NFS mount.

4

FYI - I can’t get this test case to work. The contents of the underlying “/opt/data/datavol1” volume in the “nfs-volumes” container shows through to the “worker1” container run with “–volumes-from nfs-volumes”. The NFS mount contents is visible mounted on “/opt/data/datavol1” in the nfs-volume container before and after the creation of the worker container.

However, if I then umount /opt/data/datavol1 in the nfs-volume container, and “touch /opt/data/datavol1/hello_world.txt”, the “hello_world.txt” file is immediately visible in the worker1 container.

Has anyone got the @imikushin’s example to work on the docker versions on rancherOS 0.3.3?

[rancher@rancher03 ~]$ uname -a
Linux rancher03 3.19.8-ckt1-rancher #1 SMP Thu Jun 25 12:46:25 UTC 2015 x86_64 GNU/Linux

5

Michael, I’ll try to dig into this a bit later. Meanwhile, can you provide your nfs-client details (sources, Dockerfile)?

1 Like
6

I kept it simple:

docker run --privileged=true -it --name nfs-client --net=host -v /opt/data/datavol1 ubuntu bash
nfs-client# apt-get install nfs-common -y
nfs-client# mount -o nolock $SERVER:$SHARE /opt/data/datavol1
nfs-client# ls /opt/data/datavol1
[the files on $SERVER]

7

@imikushin Is this working now or am I going to waste time trying this suggested yml? I am on 0.4.4.

Please advise.

Thank you.

8

@2devnull Are you on v0.4.0? We don’t have a v0.4.4 for RancherOS. :smile:

Here are the official docs on supporting NFS. http://docs.rancher.com/os/configuration/storage/

9

Yes, 0.4 sorry.

Yes, I have moved on to those docs in your recent links, but I am a bit confused on the “Custom Storage Container”. I see it requires an image. What do I put in that image? Do I mount the shares in that image as was done via console in “Modifying Console”?

10

I would like some help in understanding what needs to be in this image:
image: example/custom-nfs-image

Can someone add some more info to the NFS storage documentation on exactly how the NFS server is mapped for a Custom Storage Container, is it fstab, netshare plugin, etc.???

Thank you.

11

@2devnull here is an example nfs client image:

FROM       rhel-7.1

# USAGE
# $ docker build -t nfs-client .
# $ docker run -it --privileged=true --net=host -v /mnt/nfs-1 -e SERVER=192.168.0.3 -e SHARE=/export/users nfs-client
#    or detached:
#       $ docker run -itd --privileged=true --net=host -v /mnt/nfs-1 -e SERVER=192.168.0.3 -e SHARE=/export/users nfs-client
#    or with some more options:
#       $ docker run -it \
#             --name nfs-vols \
#             --restart=always \
#             --privileged=true \
#             --net=host \
#             -v /:/mnt/host
#             -e SERVER=192.168.0.104 \
#             -e SHARE=/srv/nfs4/test \
#             -e MOUNT_OPTIONS=ro \
#             -e FSTYPE=nfs4 \
#             -e MOUNTPOINT=/mnt/host/mnt/nfs-1 \
#                nfs-client

ENV FSTYPE nfs
ENV MOUNT_OPTIONS nolock
ENV MOUNTPOINT /mnt/nfs-1

RUN yum -y install nfs-utils nfs-utils-lib portmap

CMD mkdir -p "$MOUNTPOINT" && mount -t "$FSTYPE" -o "$MOUNT_OPTIONS" "$SERVER:$SHARE" "$MOUNTPOINT" && mount && cat
12

Ah…great man. Thank you!

So, once I have this as the image pushed to a repo, I would just use this image in this location of cloud-config:
example/custom-nfs-image

and leave the rest the same?

What would be the share exposed to the container that would then use it, this: /mnt/nfs-1 ?

Thanks…trying to string these pieces together and come up with a mental tutorial.

13

I made https://github.com/flaccid/docker-nfs-client so you would add some environment variables to the yaml for the server and share but I’m trying to test this with rancheros v0.4.0. Seems to be some issue with the vagrant image. @imikushin should be able to help when next available (system-docker returns FATA[0000] no such file or directory).

14

Raised https://github.com/rancher/os/issues/641.

15

@flaccid Thanks! I’m investigating

16

OK, I’m stuck on this NFS thing. It appears on boot up, RancherOS cannot get the custom nfs client image from Docker Hub or any other registry.

` # Your custom image
image: 2devnull/nfs-client

time=“2015-11-19T22:28:33.119924095Z” level=error msg="Handler for POST /v1.15/containers/create returned error: No such image: 2devnull/nfs-client:latest"
time=“2015-11-19T22:28:33.119952683Z” level=error msg=“HTTP Error” err=“No such image: 2devnull/nfs-client:latest” statusCode=404 `

17

Try to add label io.rancher.os.after: wait-for-network to the container

18

There has been a lot of progress on https://github.com/docker/docker/issues/4213 and I’ve updated my https://github.com/flaccid/docker-nfs-client. Would be curious to know if there are any issues using it on the current RancherOS.

19

Hi, I’m on v1.0.2 and interested in NFS mount on the RancherOS host.

The use case:

I have Rancher Server running in the cloud. Some of the connected hosts are running on premise and some in the cloud. I have NFS server that is running on prem and I want my hosts that are running on prem to be able to mount NFS shares.
I can’t use the Rancher NFS storage stack from catalog because Rancher Server can’t see the NFS server. I have CentOS and RancherOS hosts running on prem.
So, what exactly needs to be done to make it (NFS shares mounted to the RancherOS host) work?

20

nevermind… after i found this Mounting a disk
and this https://github.com/rancher/os/issues/1620 i’m able to mount nfs share (don’t forget to pre-create the mount directory!)