Connecting convoy gluster to external GlusterFS cluster

Convoy
1

I have an existing GlusterFS cluster. Its running on (for example) gfs1.domain.com and gfs2.domain.com. I want to setup Convoy-Gluster from the catalog to connect to it.

I created and externals service called GlusterFS. (tried with IP and with host name)

Pasted image1204×118 17.9 KB

I configured ConvoyGluster like this:

Pasted image930×145 13.1 KB

These are my logs:

5/16/2016 11:20:58 AMWaiting for metadata
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=info msg="Execing [/usr/bin/nsenter --mount=/proc/932/ns/mnt -F -- /var/lib/docker/aufs/mnt/ccab49dd0b97360cc3c79657cb694afb39135efbbb37c310f1881c0a59f1491c/var/lib/rancher/convoy-agent/share-mnt --stage2 /var/lib/rancher/convoy/convoy-gluster-4728c7c6-6c56-4cc6-a55e-65798548a2fb -- /launch volume-agent-glusterfs-internal]"
5/16/2016 11:20:58 AMWaiting for metadata
5/16/2016 11:20:58 AMRegistering convoy socket at /var/run/convoy-convoy-gluster.sock
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=info msg="Listening for health checks on 0.0.0.0:10241/healthcheck"
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=info msg="Got: drivers [glusterfs]"
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=info msg="Got: driver-opts [glusterfs.defaultvolumepool=vol1 glusterfs.servers=glusterfs]"
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=info msg="Got: root /var/lib/rancher/convoy/convoy-gluster-4728c7c6-6c56-4cc6-a55e-65798548a2fb"
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=info msg="Launching convoy with args: [--socket=/host/var/run/convoy-convoy-gluster.sock daemon --drivers=glusterfs --driver-opts=glusterfs.defaultvolumepool=vol1 --driver-opts=glusterfs.servers=glusterfs --root=/var/lib/rancher/convoy/convoy-gluster-4728c7c6-6c56-4cc6-a55e-65798548a2fb]"
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=debug msg="Creating config at /var/lib/rancher/convoy/convoy-gluster-4728c7c6-6c56-4cc6-a55e-65798548a2fb" pkg=daemon
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=debug msg= driver=glusterfs driver_opts=map[glusterfs.defaultvolumepool:vol1 glusterfs.servers:glusterfs] event=init pkg=daemon reason=prepare root="/var/lib/rancher/convoy/convoy-gluster-4728c7c6-6c56-4cc6-a55e-65798548a2fb"
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=debug msg="Volume vol1 is being mounted it to /var/lib/rancher/convoy/convoy-gluster-4728c7c6-6c56-4cc6-a55e-65798548a2fb/glusterfs/mounts/vol1, with option [-t glusterfs]" pkg=util
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=debug msg="Cleaning up environment..." pkg=daemon
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=error msg="Failed to execute: mount [-t glusterfs glusterfs:/vol1 /var/lib/rancher/convoy/convoy-gluster-4728c7c6-6c56-4cc6-a55e-65798548a2fb/glusterfs/mounts/vol1], output Mount failed. Please check the log file for more details.\n, error exit status 1"
5/16/2016 11:20:58 AM{
5/16/2016 11:20:58 AM	"Error": "Failed to execute: mount [-t glusterfs glusterfs:/vol1 /var/lib/rancher/convoy/convoy-gluster-4728c7c6-6c56-4cc6-a55e-65798548a2fb/glusterfs/mounts/vol1], output Mount failed. Please check the log file for more details.\n, error exit status 1"
5/16/2016 11:20:58 AM}
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=info msg="convoy exited with error: exit status 1"
5/16/2016 11:20:58 AMtime="2016-05-16T18:20:58Z" level=info msg=Exiting.

I can confirm that this command works successfully: sudo mount -t glusterfs gfs1.domain.com:/vol1 /mnt/vol1

My theory is that the external service is not working correctly, but I don’t know what the fix would be.

Any help would be appreciated.

2

So here is what I ended up doing.

We love Rancher so much we forget that our entire infrastructure does not have to be built with containers. First of all I should mention that I am using AWS. Since I have my GlusterFS environment in place already and my Rancher hosts are being created with an AutoScaling group on AWS the setup was actually quite easy.

This is my “User Data” for my Launch Configuration. I added the last 3 lines.

Now my GlusterFS Volume is mounted on every RancherPool Host. :smile:

 #!/bin/bash
sudo perl -pi -e 's/^#?Port 22$/Port 2222/' /etc/ssh/sshd_config
sudo service ssh restart
sudo docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher -e CATTLE_HOST_LABELS=RancherPool=yes rancher/agent:v1.0.1 http://rch.mydomain.com:8080/v1/scripts/D67D5110CBA5A0577D90:1463691600000:50adKAtfd1aWesWH9BgSdi9GBmU
sudo apt-get install -y glusterfs-client
sudo mkdir /app
sudo mount -t glusterfs glusterfs1.mydomain.com:vol1 /app