Launching rancher-agent 1.0.2 issues on RHEL 7.2

Rancher 1.x
1

I’m not sure this is specific to rancher-agent or docker in general but when I launch the rancher-agent with privileged mount to /var/run any of the host’s tmpfs mounts become locked from writing to and the container won’t start.

Also any process that needs to write to any of the tmpfs mounted filesystems don’t work on the host.

e.g. /run, /var/run/, /tmp

A reboot is required to fix this. I’m not exactly sure what happens as this is what I see in the agent’s container log right at the time it happens:

INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.0.2
time=“2016-07-15T12:45:12Z” level=fatal msg=“Error response from daemon: rpc error: code = 2 desc = “open /run/containerd/da29a351b101bc35c5d14aacb59ba210123f6e9a68bdf44bcbff6ac9efeea502/init/pid: no such process””

I believe that error is caused because something is happening on the host that effects anything mounted using /dev/shm (tmpfs). A reboot is required for anything to corectly with tmpfs mounted filesystems again.

I’m not even sure how I would start to debug this.

I have used multiple different version of 3.10.x kernel and docker-engine 1.10.3, 1.11.1 and 1.11.2 and all exhibit the same behavior.

This is on RHEL 7.2 host.

2

Solved.

It was sav (Sophos AntiVirus) agent running on the server and trying to scan the thin-mount lvm device that devicemapper was using to write images to. Disabling the sav software and/or re-configuring to ignore that device has fixed the issue.

Thank you.