Hi…Can anybody provide a link/doc for configuring convoy-nfs? I am lost at this and is stuck - The convoy-nfs container is initializing for 4 hours now.
What’s your Docker version?
Also, anything shows in Convoy log?
Also, can you create a new container with volume and convoy-nfs driver? We’ve received multiple reports on this issue but it seems just our status detection is off. Convoy-nfs is already up in some cases, though not sure about if that’s the case here.
Hi Yasker,
The version is :-
root@infra-server:~# docker --version
Docker version 1.6.2, build 7c8fca2
root@infra-server:~#
Logs for convoy:-
Waiting for metadata.
time=“2016-09-07T06:41:00Z” level=info msg=“Execing [/usr/bin/nsenter --mount=/proc/643/ns/mnt -F – /var/lib/docker/aufs/mnt/a7dca2e9e74ed0a893758f41cc3c959af5a35a7eaef6268ef98f9945a05a4353/var/lib/rancher/convoy-agent/share-mnt --stage2 /var/lib/rancher/convoy/convoy-nfs-3a6cb228-eedb-4d42-bc91-3af4adb80185 – /launch volume-agent-nfs-internal 643]“
time=“2016-09-07T12:11:00+05:30” level=warning msg=“exit status 1"
time=“2016-09-07T12:11:00+05:30” level=error msg=“Failed to start (pid 994) &libcontainer.Process{Args:[]string{”/launch”, “volume-agent-nfs-internal”, “643”}, Env:[]string{“CATTLE_CONFIG_URL=http://192.168.0.122:8080/v1”, “HOSTNAME=a7dca2e9e74e”, “CATTLE_ACCESS_KEY=E96F700349681AAE6C14”, “CATTLE_AGENT_INSTANCE_AUTH=Basic RTk2RjcwMDM0OTY4MUFBRTZDMTQ6aFFtdXoxSHNSdlB5eDJGVlh2SlY3M1ZaVDhRREpHa1RKREdRSE5FWg==”, “CATTLE_SECRET_KEY=hQmuz1HsRvPyx2FVXvJV73VZT8QDJGkTJDGQHNEZ”, “PATH=/var/lib/rancher/convoy-agent:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”, “PWD=/”, “SHLVL=1”, “HOME=/root”, “CATTLE_URL=http://192.168.0.122:8080/v1”, “=/var/lib/rancher/convoy-agent/share-mnt"}, User:“0:0”, Cwd:"", Stdin:(*os.File)(0xc82002a110), Stdout:(*os.File)(0xc82002a128), Stderr:(*os.File)(0xc82002a138), ExtraFiles:[]*os.File(nil), consolePath:"", Capabilities:[]string(nil), ops:libcontainer.processOperations(nil)}: [8] System error: operation not permitted"
time=“2016-09-07T12:11:00+05:30” level=fatal msg="[8] System error: operation not permitted"
Waiting for metadata
time=“2016-09-07T06:41:22Z” level=info msg="Execing [/usr/bin/nsenter --mount=/proc/643/ns/mnt -F – /var/lib/docker/aufs/mnt/a7dca2e9e74ed0a893758f41cc3c959af5a35a7eaef6268ef98f9945a05a4353/var/lib/rancher/convoy-agent/share-mnt --stage2 /var/lib/rancher/convoy/convoy-nfs-3a6cb228-eedb-4d42-bc91-3af4adb80185 – /launch volume-agent-nfs-internal 643]"
time=“2016-09-07T12:11:22+05:30” level=warning msg=“exit status 1"
time=“2016-09-07T12:11:22+05:30” level=error msg=“Failed to start (pid 2098) &libcontainer.Process{Args:[]string{”/launch”, “volume-agent-nfs-internal”, “643”}, Env:[]string{“CATTLE_CONFIG_URL=http://192.168.0.122:8080/v1”, “HOSTNAME=a7dca2e9e74e”, “CATTLE_ACCESS_KEY=E96F700349681AAE6C14”, “CATTLE_AGENT_INSTANCE_AUTH=Basic RTk2RjcwMDM0OTY4MUFBRTZDMTQ6aFFtdXoxSHNSdlB5eDJGVlh2SlY3M1ZaVDhRREpHa1RKREdRSE5FWg==”, “CATTLE_SECRET_KEY=hQmuz1HsRvPyx2FVXvJV73VZT8QDJGkTJDGQHNEZ”, “PATH=/var/lib/rancher/convoy-agent:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”, “PWD=/”, “SHLVL=1”, “HOME=/root”, “CATTLE_URL=http://192.168.0.122:8080/v1”, "=/var/lib/rancher/convoy-agent/share-mnt”}, User:“0:0”, Cwd:””, Stdin:(*os.File)(0xc82002a0f8), Stdout:(*os.File)(0xc82002a110), Stderr:(*os.File)(0xc82002a120), ExtraFiles:[]*os.File(nil), consolePath:"", Capabilities:[]string(nil), ops:libcontainer.processOperations(nil)}: [8] System error: operation not permitted"
time=“2016-09-07T12:11:22+05:30” level=fatal msg="[8] System error: operation not permitted"
I can create new container with volume (-v /opt:/opt/software). It works fine. But convoy nfs container is exiting and restarting. After some time it get stopped. I have tried to remove the container and re-create the convoy-nfs. But the same issue persists.
The OS is Ubuntu 16.04 LTS and I am running as root.
Did you remember to set privileged: true on the convoy-nfs agent?
Also, my experience is that the convoy-nfs agent asks the OS kernel to do the actual mounting. In case you have made a mistake and asked the kernel to e.g. mount a folder from a non-existent server, then redeployments won’t succeed until the host is rebooted.
Hi soren,
Yes…I checked that privileged is true for the convoy-nfs agent. I have installed and configured nfs server in the host system and it is accessible.
Hello,
Can you explain little bit more? I know how you feel 
I was in the same situations and I have it very fresh now.
What did you use for create the NFS server first? Did you use an image for the NFS server or are you using an external NFS?
I found this guide very instering but need some changes: http://rancher.com/setting-shared-volumes-convoy-nfs/
It’s a relatively new guide and it’s very up to date.
I hope it helps you!
Hi danipolo,
Here below is the scenario :-
I installed docker in 192.168.0.122 system. I installed single-node rancher server in it. I added a different host (192.168.0.182) as rancher agent. I created 2 nfs shares from both system (122 and 182). From the web portal of rancher server I configured convoy-nfs (From Catalog–convoy-nfs). I gave 192.168.0.122 as nfs server address and /home as mount directory. I left mount options blank as I do not know what that is. Once created it is trying to create a container in the both host. But the containers remains in restarting mode.
Please let me know if this is not the process?
Did you try to follow this guide?
Please provide the convoy-nfs log and the NFS server log.
/home as mount directory? Here you should fill the directory you specified in /etc/exports. So you’ve exported /home in NFS?
Hi Yasker,
Yes…I had exported /home and also verified from other system that it is working fine before configuring convoy. Here is what my exports file look like :-
/home 192.168.0.0/24(rw,sync) 172.17.0.0/16(rw,sync) 10.42.0.0/16(rw,sync)
My Convoy-NFS is stuck at “initializing”. This is the log:
9/25/2016 5:48:58 PMWaiting for metadata9/25/2016 5:49:00 PMtime="2016-09-25T15:49:00Z" level=info msg="Execing [/usr/bin/nsenter --mount=/proc/21890/ns/mnt -F -- /var/lib/docker/aufs/mnt/92a90b8886fcc95fa179c737ce16d7da267e72904ef3e2a63dc8629d8f217f83/var/lib/rancher/convoy-agent/share-mnt --stage2 /var/lib/rancher/convoy/convoy-nfs-9cc7a338-0aaa-4508-9a5b-538729433d95 -- /launch volume-agent-nfs-internal 21890]"9/25/2016 5:49:00 PMRegistering convoy socket at /var/run/convoy-convoy-nfs.sock9/25/2016 5:49:00 PMMounting at: /var/lib/rancher/convoy/convoy-nfs-9cc7a338-0aaa-4508-9a5b-538729433d95/mnt9/25/2016 5:49:00 PMMounting nfs. Command: mount -t nfs -o proto=tcp,port=2049,nfsvers=4 10.42.118.232:/ /var/lib/rancher/convoy/convoy-nfs-9cc7a338-0aaa-4508-9a5b-538729433d95/mnt9/25/2016 5:51:05 PMmount.nfs: Connection timed out
Any help?? It seems as the mount -t command is not executed due to lack or permissions or closed port!
UPDATE: I found a but with dockerized nfs-server. Check out https://github.com/cpuguy83/docker-nfs-server/issues/10