Convoy-gluster stack doesn't start

Rancher 1.x
1

I cannot start the convoy-gluster stack from the catalog. I have successfully started the GlusterFS service from the catalog and I configured the convoy-gluster stack to connect to it.

However, when I start the stack, the convoy-gluster-storagepool container goes to a “Degraded” state within a few seconds. The logs from the container are

12/1/2015 6:00:46 PMtime="2015-12-01T23:00:46Z" level=info msg="Listening for health checks on 0.0.0.0:10241/healthcheck"
12/1/2015 6:00:51 PMtime="2015-12-01T23:00:51Z" level=debug msg="storagepool event [da8f2997-a1e9-40bd-9191-a79ff4ff356c]"
12/1/2015 6:04:48 PMWaiting for metadata.
12/1/2015 6:04:54 PMtime="2015-12-01T23:04:54Z" level=info msg="Listening for health checks on 0.0.0.0:10241/healthcheck"
12/1/2015 6:04:59 PMtime="2015-12-01T23:04:59Z" level=debug msg="storagepool event [da8f2997-a1e9-40bd-9191-a79ff4ff356c]"
12/1/2015 6:05:14 PMWaiting for metadata.
12/1/2015 6:05:21 PMtime="2015-12-01T23:05:21Z" level=info msg="Listening for health checks on 0.0.0.0:10241/healthcheck"
12/1/2015 6:05:26 PMtime="2015-12-01T23:05:26Z" level=debug msg="storagepool event [da8f2997-a1e9-40bd-9191-a79ff4ff356c]"
12/1/2015 6:09:24 PMWaiting for metadata.
12/1/2015 6:09:31 PMtime="2015-12-01T23:09:31Z" level=info msg="Listening for health checks on 0.0.0.0:10241/healthcheck"
12/1/2015 6:09:36 PMtime="2015-12-01T23:09:36Z" level=debug msg="storagepool event [da8f2997-a1e9-40bd-9191-a79ff4ff356c]"
2

How many hosts do you have in your Rancher set up? Are they able to communicate with each other? If you exec into the network agent, ping the IPs of the network agent on the other hosts.

It seems as if the healthcheck is failing, which is usually because cross host communication isn’t working.

Also, can you provide what OS you are running on your hosts?

3

Would you be able to share more information about the issue that you were running into with actually deploying convoy-gluster? What would be most helpful is which convoy-gluster containers were not starting and the full logs from one such container.

4

I think I figured this out. What I didn’t realize was that you had to set convoy.gluster=true labels on your hosts. I think the problem was that because I didn’t have the labels, Rancher did not find any suitable hosts to deploy the containers to.

1 Like
5

Great! I’m glad you figured it out. Apologies as I had just figured out that might be your issue yesterday, but forgot to follow up on this thread.