RancherOS 1.5 RC - NFS Mount

bott0r · December 4, 2018, 3:12pm

Hey Ho, quite the beginner here, I am using RancherOS for homelab use.

In previous versions I mounted my NFS shares to different folders in RancherOS/mnt/ using the nfs client, my nfs.yml looked like this, and it worked just fine. In my containers i just added volumes /mnt/data:/mnt/data and so on

#/var/lib/rancher/conf/cloud-config.d/nfs.yml
write_files:
  - path: /etc/rc.local
    permissions: "0755"
    content: |
      #!/bin/bash
      [ ! -e /usr/bin/docker ] && ln -s /usr/bin/docker.dist /usr/bin/docker

rancher:
  services:
    nfs:
      image: d3fk/nfs-client
      labels:
        io.rancher.os.after: console, preload-user-images
        io.rancher.os.scope: system
      net: host
      privileged: true
      restart: always
      volumes:
        - /usr/bin/iptables:/sbin/iptables:ro
        - /mnt/data:/mnt/data:shared
        - /mnt/data2:/mnt/data2:shared
      environment:
        SERVER: 192.168.0.2
        SHARE: /mnt/volume01/data
        MOUNTPOINT: /mnt/data

#cloud-config
mounts:
  - ["192.168.0.2:/mnt/volume02/data2", "/mnt/data2", "nfs", ""]

after setting up a new host i thought i’d try 1.5RC which installed quite quickly and easy. Great.
Now I wanted to mount my NFS shares and tried the same thing, which ultimately did not work. Then i noticed that normal sudo mount commands work perfectly fine. So here is my question if i’d like to mount an nfs share to my host how would i do it?

#workd out of the box
sudo mount -o nfsvers=4,port=2049,nolock,proto=tcp -t nfs 192.168.0.2:/mnt/volume01/data /mnt/data

bott0r · December 4, 2018, 4:49pm

It works like somewhat, if anyone else has an idea

#/var/lib/rancher/conf/cloud-config.d/nfs-data.yml  first part working

write_files:
  - path: /etc/rc.local
    permissions: "0755"
    content: |
      #!/bin/bash
      [ ! -e /usr/bin/docker ] && ln -s /usr/bin/docker.dist /usr/bin/docker
rancher:
  services:
    nfs:
      image: walkerk1980/rancher-nfs-client
      net: host
      privileged: true
      restart: always
      volumes:
        - /usr/bin/iptables:/sbin/iptables:ro
        - /mnt/data:/mnt/data:shared
      environment:
        SERVER: 192.168.0.2
        MOUNT_OPTIONS: port=2049,nolock
        SHARE: /mnt/volume01/data
        MOUNTPOINT: /mnt/data

What did not work is creating a simple start.sh script as suggested somewhere.

#cloud-config
write_files:
  - path: /opt/rancher/bin/start.sh
    permissions: "0755"
    owner: root
    content: |
      #!/bin/bash
      sudo mount -t nfs4 192.168.0.2:/mnt/volume01/data /mnt/data

also did not work, the folders get created in /mnt but nothing is mounted.

bott0r · December 6, 2018, 1:02pm

Coming back to this. Without a nfs client i can just mount via a simple line:
sudo mount -t nfs4 192.168.0.2:/mnt/volume01/data /mnt/data

However when i translate thet line to a cloudconfig with either:

#/var/lib/rancher/conf/cloud-config.d/z.yml
mounts:
- ["192.168.0.2:/mnt/volume01/data", "/mnt/data", "nfs4", ""]

or with

ros config set mounts '[["192.168.0.2:/mnt/volume01/data", "/mnt/data1", "nfs4", ""]]'

it just doesnt work, what am i missing, i crawled by now through the documentation that is available or cached 404 pages, along side closed github issues. does anyone have a pointer here?

Jason-ZW · December 10, 2018, 6:06am

@bott0r

github.com/rancher/os

Formatting disk

opened 07:44PM - 07 Jun 18 UTC

closed 04:03AM - 11 Jun 18 UTC

drpebcak

kind/question

**RancherOS Version: (ros os version)** v1.4.0 **Where are you running Ranch…erOS? (docker-machine, AWS, GCE, baremetal, etc.)** AWS I am attempting to use cloud-config to format and mount the ephemeral instance store in a new m5d instance: ``` #cloud-config bootcmd: - [ mkfs.ext2, /dev/nvme1n1 ] mounts: - [ /dev/nvme1n1, /storage, ext2, "" ] rancher: docker: extra_args: - "-g" - /storage ``` When the instance starts up, `/storage` exists, and user-docker has put its state there. However, `nvme1n1` is not formatted, and there is no mount on `/storage` Any ideas? I have been unable to find any logs or errors that show what is happening during the bootstrap.

Try using runcmd:

#cloud-config
runcmd:
  - [ mkfs.ext2, /dev/nvme1n1 ]
  - [ mkdir, -p, /storage]
  - [ mount, -t, ext2, /dev/nvme1n1, /storage ]

bott0r · December 10, 2018, 7:01am

I tried various combinations and it wont work for me on 1.5.
this is the last i tried.

#cloud-config
runcmd:
- [ mkdir, -p, /mnt/storage]
- [ mkdir, -p, /mnt/data]
- [ "mount", "-t", "nfs4","192.168.0.2:/mnt/volume01/data", "/mnt/data" ]

Folders get created, but nothing will be mounted, if i take the quotation marks away ros config validate will complain. anything else i could try?

Jason-ZW · December 11, 2018, 8:42am

I use RancherOS 1.4.2 rancheros-vmware.iso successfully mount directory form my nfs server.
RancherOS mount command is build from busybox, not the GNU mount command, so there maybe some different.RancherOS has no mount.nfs & mount.nfs4 which installed by nfs-common by default, so without mount.nfs... user need to add options.

#cloud-config
mounts:
- ["192.168.1.109:/mnt/data", "/mnt/test", "nfs4", "nolock,proto=tcp,addr=192.168.1.109"]

output:

[root@rancheros-nfs docker]# ls /mnt/test
1.txt

bott0r · December 11, 2018, 10:43am

hey,
I am on RancherOS 1.5-RC-1 and thes rancheros.iso Releases · rancher/os · GitHub

These are the file contents for the mount, the test folder is created after a reboot but its still not mounted. I tried the command sudo mount -o nolock -t nfs4 192.168.0.2:/mnt/volume01/data /mnt/data

It shows the content:

[rancher@beta /]$ sudo mount -o nolock -t nfs4 192.168.0.2:/mnt/volume01/data /mnt/data

[rancher@beta /]$ ls /mnt/data

Physics Complete ToCheckToDelete docker friends important sorted-data
[rancher@beta /]$

when using a file in cloud-config.d with the content it just wont show. I am wondering if the issue is with rancher1.5

#/var/lib/rancher/conf/cloud-config.d/z.yml
mounts:
- ["192.168.0.2:/mnt/volume01/data", "/mnt/test", "nfs4", "nolock,proto=tcp,addr=192.168.0.2"]

bott0r · December 11, 2018, 1:06pm

It worked for me on 1.4.2 but not 1.5

Jason-ZW · December 12, 2018, 5:15am

I also try on 1.5.0-rc1 rancheros.iso, the result is success.

[root@rancher rancher]# ros -v
version v1.5.0-rc1 from os image rancher/os:v1.5.0-rc1

[root@rancher rancher]# ros c export
mounts:
- - 192.168.1.79:/mnt/data
  - /mnt/test
  - nfs4
  - ""
rancher:
  environment:
    EXTRA_CMDLINE: /init
  services_include:
    open-vm-tools: true
  state:
    dev: LABEL=RANCHER_STATE
    wait: true
ssh_authorized_keys: []

[root@rancher rancher]# df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                   7.4G    773.4M      6.3G  11% /
tmpfs                   930.9M         0    930.9M   0% /dev
tmpfs                   986.4M         0    986.4M   0% /sys/fs/cgroup
/dev/sda1                 7.4G    773.4M      6.3G  11% /opt
/dev/sda1                 7.4G    773.4M      6.3G  11% /mnt
/dev/sda1                 7.4G    773.4M      6.3G  11% /home
none                    986.4M    876.0K    985.5M   0% /run
/dev/sda1                 7.4G    773.4M      6.3G  11% /media
/dev/sda1                 7.4G    773.4M      6.3G  11% /etc/resolv.conf
/dev/sda1                 7.4G    773.4M      6.3G  11% /etc/selinux
/dev/sda1                 7.4G    773.4M      6.3G  11% /usr/lib/firmware
none                    986.4M    876.0K    985.5M   0% /var/run
/dev/sda1                 7.4G    773.4M      6.3G  11% /etc/logrotate.d
/dev/sda1                 7.4G    773.4M      6.3G  11% /usr/lib/modules
/dev/sda1                 7.4G    773.4M      6.3G  11% /var/log
/dev/sda1                 7.4G    773.4M      6.3G  11% /usr/sbin/iptables
/dev/sda1                 7.4G    773.4M      6.3G  11% /etc/docker
/dev/sda1                 7.4G    773.4M      6.3G  11% /etc/hosts
devtmpfs                930.9M         0    930.9M   0% /host/dev
shm                      64.0M         0     64.0M   0% /host/dev/shm
/dev/sda1                 7.4G    773.4M      6.3G  11% /etc/hostname
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/sda1                 7.4G    773.4M      6.3G  11% /usr/bin/ros
/dev/sda1                 7.4G    773.4M      6.3G  11% /var/lib/kubelet
/dev/sda1                 7.4G    773.4M      6.3G  11% /var/lib/docker
/dev/sda1                 7.4G    773.4M      6.3G  11% /var/lib/boot2docker
/dev/sda1                 7.4G    773.4M      6.3G  11% /var/lib/rancher
/dev/sda1                 7.4G    773.4M      6.3G  11% /usr/share/ros
/dev/sda1                 7.4G    773.4M      6.3G  11% /usr/bin/system-docker-runc
/dev/sda1                 7.4G    773.4M      6.3G  11% /usr/bin/system-docker
/dev/sda1                 7.4G    773.4M      6.3G  11% /var/lib/m-user-docker
/dev/sda1                 7.4G    773.4M      6.3G  11% /etc/ssl/certs/ca-certificates.crt.rancher
/dev/sda1                 7.4G    773.4M      6.3G  11% /var/lib/rancher/conf
/dev/sda1                 7.4G    773.4M      6.3G  11% /var/lib/rancher/cache
devtmpfs                930.9M         0    930.9M   0% /dev
shm                      64.0M         0     64.0M   0% /dev/shm
192.168.1.79:/mnt/data
                         18.6G      1.5G     16.1G   9% /mnt/test

bott0r · December 12, 2018, 6:29am

@Jason-ZW

 [root@beta rancher]# ros c export
hostname: beta
mounts:
- - 192.168.0.2:/mnt/volume01/data
  - /mnt/test
  - nfs4
  - ""
rancher:
  environment:
    EXTRA_CMDLINE: /init
  network:
    dns:
      nameservers:
      - 192.168.0.1
      - 192.168.1.1
    interfaces:
      eth0:
        dhcp: true
        mtu: 1500
  state:
    dev: LABEL=RANCHER_STATE
    wait: true
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2E


[root@beta test]# ros -v
version v1.5.0-rc1 from os image rancher/os:v1.5.0-rc1

[root@beta test]#  df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                  52.1G     11.6G     37.8G  23% /
tmpfs                     1.8G         0      1.8G   0% /dev
tmpfs                     1.9G         0      1.9G   0% /sys/fs/cgroup
none                      1.9G      1.2M      1.9G   0% /run
/dev/sda1                52.1G     11.6G     37.8G  23% /home
/dev/sda1                52.1G     11.6G     37.8G  23% /mnt
/dev/sda1                52.1G     11.6G     37.8G  23% /opt
/dev/sda1                52.1G     11.6G     37.8G  23% /media
/dev/sda1                52.1G     11.6G     37.8G  23% /etc/resolv.conf
/dev/sda1                52.1G     11.6G     37.8G  23% /usr/sbin/iptables
/dev/sda1                52.1G     11.6G     37.8G  23% /etc/logrotate.d
none                      1.9G      1.2M      1.9G   0% /var/run
/dev/sda1                52.1G     11.6G     37.8G  23% /etc/docker
/dev/sda1                52.1G     11.6G     37.8G  23% /usr/lib/modules
devtmpfs                  1.8G         0      1.8G   0% /host/dev
shm                      64.0M         0     64.0M   0% /host/dev/shm
/dev/sda1                52.1G     11.6G     37.8G  23% /usr/lib/firmware
/dev/sda1                52.1G     11.6G     37.8G  23% /etc/selinux
/dev/sda1                52.1G     11.6G     37.8G  23% /etc/hosts
/dev/sda1                52.1G     11.6G     37.8G  23% /var/log
/dev/sda1                52.1G     11.6G     37.8G  23% /etc/hostname
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/sda1                52.1G     11.6G     37.8G  23% /usr/share/ros
/dev/sda1                52.1G     11.6G     37.8G  23% /var/lib/docker
/dev/sda1                52.1G     11.6G     37.8G  23% /var/lib/m-user-docker
/dev/sda1                52.1G     11.6G     37.8G  23% /usr/bin/ros
/dev/sda1                52.1G     11.6G     37.8G  23% /var/lib/boot2docker
/dev/sda1                52.1G     11.6G     37.8G  23% /var/lib/kubelet
/dev/sda1                52.1G     11.6G     37.8G  23% /var/lib/rancher
/dev/sda1                52.1G     11.6G     37.8G  23% /usr/bin/system-docker
/dev/sda1                52.1G     11.6G     37.8G  23% /usr/bin/system-docker-runc
/dev/sda1                52.1G     11.6G     37.8G  23% /etc/ssl/certs/ca-certificates.crt.rancher
/dev/sda1                52.1G     11.6G     37.8G  23% /var/lib/rancher/conf
/dev/sda1                52.1G     11.6G     37.8G  23% /var/lib/rancher/cache
devtmpfs                  1.8G         0      1.8G   0% /dev
shm                      64.0M         0     64.0M   0% /dev/shm
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/ddff1130ae1e5a49331a523d6b789cfef97668d574ce8f1f045ea32f413af198/merged
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/7b38e0e71b3c58c44cc12fa5d30c5a8543414ba1be5a7989d808b867b839fc5e/mounts/shm
/dev/sda1                52.1G     11.6G     37.8G  23% /var/lib/rancher/volumes
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/ab0494408fd01a225f4251edc007ad4f55650658b2625be26cbbfdefe6482d3f/merged
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/42f511b62ba51da7a725a48a7a6190a6b2f3831bbc01677cc3f5580c65d27442/mounts/shm
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/84efbfdd41f84844d496ea9513d66976c818ab7336838ac60df2bc66fe2f32c6/merged
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/0c300decfc6c941cc98cf762c36d3e0ff9db15ee1c6102b222d03a684bed67b9/merged
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/c769b6cfe2ddd9bf97a660e0493bc93698546065422be03d1716d1796b841a12/mounts/shm
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/74f7f4e1e809e452d115f1cb8cdf832e1f0bc55d4e55f17478af47bba6b2c6de/merged
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/90ebbc8234225a71609279f175ae8fed92a8d93fd3f75782c17642de0014aa02/mounts/shm
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/4456312fbed7de788eabc42351dfb09cc48eecbf84a56b665ff4df6b8c0726c8/mounts/shm
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/addf7119e5ed1d3f3c00e18fb20b97d3b082d9542583c59d593adbd4330b73cd/merged
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/81016c051aec4332a2406cb0e79253bd73b2cacd19a76362b7069c07f6408f96/mounts/shm
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/f23b0cc9e2b004712bcbc9612c0a47fe11007b3345df89b91cbf3ca675bccea9/merged
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/fbd778caa768083f1bbf557493b4009be057364bb06862d7f2ed3552dd5d7eef/mounts/shm
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/9092a5c435bb290411ea914ab9c6c3b1aae1b2f5a9e8a65562d2ee553ae44ad2/merged
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/45a466058290c14fa4bffc53262285f86d1b12eb9d049fcfd1bcfc7cb24df10a/mounts/shm
overlay                  52.1G     11.6G     37.8G  23% /var/lib/docker/overlay2/4fd2b9c661f262b473f5c920ab20502df59ac9999aa10df0811874fe42f2b4bd/merged
shm                      64.0M         0     64.0M   0% /var/lib/docker/containers/0af546370f74127d58f5e1e3cdda62c7d8ecab1e927ae2c6f5d1150bad1270b5/mounts/shm

I mean i am typing in the same stuff

bott0r · December 12, 2018, 6:37am

I also tried a share that doesnt have numbers, just in case but to no avail.

I have 3 Rancher bare metal system up and running 2 1.5-rc1 and 1.4.2 (mounts perfectly fine) also if i install another nfs-client it works as well as mentioned above. Any special configs i need to be aware of.

bott0r · December 12, 2018, 6:53am

Of course everything manual works perfect too:

[root@beta mnt]# sudo mount -t nfs4 192.168.0.2:/mnt/evo/VirtualMachines /mnt/test2
[root@beta mnt]# cd test2/
[root@beta test2]# ls
RawFile_RanchreOS  test.txt
[root@beta test2]#

Topic		Replies	Views
Mounting external NFS shares on RancherOS nodes RancherOS	19	20631	June 14, 2017
How to mount multiple NFS shares? RancherOS	4	3632	October 19, 2020
RancherOS and NFS mount not working, any solution? RancherOS	1	2172	December 23, 2019
How to CIFS Mount? RancherOS	5	11865	August 29, 2021
Unable to use rancher-nfs to mount remote nfs share to container Rancher 1.x	1	3661	September 8, 2017

RancherOS 1.5 RC - NFS Mount

Related topics