Permission Denied on Docker Command

RancherOS
1

All,

I’ve upgraded my version of RancherOS to 0.7.1 and I am experiencing a strange issue. I can no longer execute the docker command. When I attempt to run it, I get the following:

rancher@rancher:~$ docker
-bash: /usr/bin/docker: Permission denied

This appears to happen on any version after 0.5.0. I rolled back to 0.5.0 and the issue went away.

Does anyone know why this would be happening? I did some searching on the forums here but I couldn’t find anyone with the same issue.

Thanks!

2 Likes
2

Can you give some more detail on your setup (cloud-config, console, Docker version, etc…)? I don’t think we’ve encountered this before and I’m unable to reproduce.

3

I’d be happy to. Sorry for the delay.

Here is the version of docker while on v0.5.0 of rancher os:

My cloud config contains the following (it is pretty bare bones):

#cloud-config
ssh_authorized_keys:
  - ssh-rsa …
mounts:
  - ["/dev/sdb", "/mnt/media", "ext4", ""]
  - ["/dev/sbc", "/mnt/backups", "ext4", ""]

and the console I am using is Debian

I’d be happy to supply any other info you can think of that would help.

4

I run into the same issue today, after upgrading to v:0.7.1 I received this message:
root@rancher:~$ docker ps
-bash: /usr/bin/docker: Permission denied

1 Like
5

This also happens to my server when I attempt to upgrade it from rancherOS 0.5.0 to either 0.6.0 or 0.7.1.

While on version 0.5.0, it has Docker version 1.11.2, build b9f10c9

My cloud-config looks like:

hostname: rancher
rancher:
  console: ubuntu
  upgrade:
    image: rancher/os
    url: https://releases.rancher.com/os/releases.yml

It seems that the docker container is constantly restarting:

root@rancher:~# docker ps
bash: /usr/bin/docker: Permission denied
root@rancher:~# ls -la /usr/bin/docker
lrwxrwxrwx 1 root root 20 Jun 28  2016 /usr/bin/docker -> /usr/bin/docker.dist
root@rancher:~# ls -la /usr/bin/docker.dist
-rw-r--r-- 1 root root 0 Nov 11 04:35 /usr/bin/docker.dist
root@rancher:~# system-docker ps
CONTAINER ID        IMAGE                               COMMAND                  CREATED             STATUS                         PORTS               NAMES
b98a91c75103        rancher/os-docker:1.12.3            "/usr/bin/user-docker"   6 minutes ago       Restarting (1) 2 minutes ago                       docker
82fdd5b3f29f        rancher/os-base:v0.7.1              "/usr/bin/ros entrypo"   6 minutes ago       Up 6 minutes                                       network
2c7c979e4b3e        rancher/os-base:v0.7.1              "/usr/bin/ros entrypo"   6 minutes ago       Up 6 minutes                                       ntp
476a59c8982c        rancher/os-udev:v0.7.1              "/usr/bin/ros entrypo"   6 minutes ago       Up 6 minutes                                       udev
32f9c8f20477        rancher/os-acpid:v0.7.1             "/usr/bin/ros entrypo"   7 minutes ago       Up 6 minutes                                       acpid
b20a977a7f58        rancher/os-syslog:v0.7.1            "/usr/bin/ros entrypo"   7 minutes ago       Up 6 minutes                                       syslog
2eb1ad13617b        rancher/os-ubuntuconsole:v0.5.0-3   "/usr/sbin/entry.sh /"   3 months ago        Up 6 minutes                                       console
root@rancher:~# system-docker logs b98a91c75103
time="2017-01-11T23:40:32Z" level=info msg="Starting Docker in context: console"
time="2017-01-11T23:40:32Z" level=info msg="Getting PID for service: console"
time="2017-01-11T23:40:32Z" level=info msg="console PID 1025"
time="2017-01-11T23:40:32Z" level=info msg="[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=rancher HOME=/]"
time="2017-01-11T23:40:32Z" level=info msg="Running [docker-runc exec -- 2eb1ad13617b1501934dde74106c752344eed658d14437197dcdf3109c7ebb0d env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=rancher HOME=/ docker-init daemon --group docker --log-opt max-file=2 --log-opt max-size=25m --storage-driver overlay --host unix:///var/run/docker.sock]"
.....
time="2017-01-11T23:47:57Z" level=info msg="Starting Docker in context: console"
time="2017-01-11T23:47:57Z" level=info msg="Getting PID for service: console"
time="2017-01-11T23:47:57Z" level=info msg="console PID 1025"
time="2017-01-11T23:47:57Z" level=info msg="[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=rancher HOME=/]"
time="2017-01-11T23:47:57Z" level=info msg="Running [docker-runc exec -- 2eb1ad13617b1501934dde74106c752344eed658d14437197dcdf3109c7ebb0d env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=rancher HOME=/ docker-init daemon --log-opt max-file=2 --log-opt max-size=25m --group docker --host unix:///var/run/docker.sock --storage-driver overlay]"
root@rancher:~#
6

The issue here is with persistent consoles. By default we don’t upgrade them when the rest of RancherOS is upgraded and this can cause issues. We’re slowly trying to move away from the the idea of stateful consoles (https://github.com/rancher/os/issues/1398).

For now, the fix is to recreate the console container so that’s it’s upgraded to the newest version. The best way to do this is to switch to a different console and then back.

ros console switch default
ros console switch ubuntu
1 Like
7

Aha! Awesome - thanks a bunch!