Not entirely sure how to go about debugging – but I just upgraded from 0.46.0 to 0.47.0 on GCE and tried to launch a GlusterFS Stack using the Catalog 3.7.5-rancher1.
The Service launches the first container and started it once. The second service (server) seems to be continually restarting… eventually showing the error:
State: Stopped ( : a7e401a5-8427-4042-a050-be9843ae3690 : 404 Client Error: Not Found (“Cannot start container 678ae9cd2012034a8f463d8b637c56cfb3c9e066997532289d8faef11bc02e91: error gathering device information while adding custom device “/dev/fuse”: lstat /dev/fuse: no such file or directory”))
From the Rancher UI. Any ideas as to what might help debug this issue? Thoughts on resolutions?
I have encountered this, and have resolved it down to these pitfalls
What OS you running?
What user is the docker daemon running under?
if running as a non-root user, the service does not have the permission to created the device. if you manually create the device see mknod/makedev the service then start.
My two test cases:
1 Host running RancherOS - failed to deploy, see above resolution
another host running ubuntu:14.0.4.3, scheduled fine with no issues.
There are further issues when it comes then to deploying the convoy-gluster service, and it not created and seeing volumes, but have submitted issues through to Rancher Labs.
Yep, I’m also running RancherOS v0.4.0 on hosts in GCE. Haven’t tried on other platforms yet but I’d completely believe thats an issue. @Moridinmhael did you open a ticket for the RancherOS issue anywhere? Thoughts on how to gather debug info on this to create a bug if not?
I think i have the same problem. I run rancher on Ubuntu and the agent on the same machine. It seams that convoy-gluster did not start and have no containers. How can i get out any error logs from the startup of the stack ?
Here’s the output of /var/log/docker.log in my stack.
RancherOS v0.4.1 on a KVM VM
Rancher 0.47.0
time="2015-11-27T08:17:05.956478206Z" level=error msg="HTTP Error" err="Cannot start container 46692e7091874b4dcac7f0bc1bc52b06fe4a6de224a9775faae9ad6bd0a3733e: error gathering device information while adding custom device \"/dev/fuse\": lstat /dev/fuse: no such file or directory" statusCode=500
What’s curios is, is that if I start the stack through rancher-compose it’s working ??
@hwaastad Thanks! That was the resolution to this specific issue (I guess I should have actually taken the hint from the error ). I did see the host label stuff as well – thanks for the tip.
My container get this error after some digging level=error msg=“Error while reading host info [json: cannot unmarshal object into Go value of type string]”.
As i describe above i started the service directly from the catalog but the convoy-gluster will not start but convoy-gluster-storagepool started.
FYI, we’ve made a fix for GlusterFS catalog item so it works on RancherOS and is compatible with systems that don’t have fuse loaded. The updated catalog entry is already available.
@Mikael_Svensson: If you are still having issues, please open another topic as your issue is different.