Unable to Create a RancherOS VM using docker-machine

RancherOS
1

I tried to run it under VirtualBox using docker-machine
and the instructions at:

http://rancher.com/getting-started-with-rancheros-2/

“You might notice that there are two .iso files, rancheros.iso
and machine-rancheros.iso. machine-rancheros.iso is the file you
want. This ISO has been built with special configuration for
setup with docker-machine.”

but at the webpage of RancherOS ISO machines,

https://github.com/rancher/os/releases

there is not one prefixed with “machine-” to use with docker-machine.

The article was written about an older version of RancherOS, so thinking
maybe that a separate machine-* ISO image is no longer needed, I proceeded with the non-machone-* ISO image.

But then I get:

 Error creating machine: Error checking the host: Error checking
 and/or regenerating the certs: There was an error validating
 certificates for host "192.168.100.100:2376":
 dial tcp 192.168.100.100:2376: getsockopt: connection refused

The exact same docker-create command works fine with boot2docker.iso.

Has RancherOS left behind usage with docker-machine? I’m just trying to
get started and it’s been rough. I had to give up on bare-metal install onto
vSphere for lack of doc about how to do that w/o DHCP network support,
where you cannot just ‘ssh’ the key onto the new root drive.

2

Did you look at the docs on how to launch RancherOS using docker-machine?

http://docs.rancher.com/os/running-rancheros/workstation/docker-machine/

We removed the machine-rancheros.iso and the latest OS version (v0.4.4) supports launching with docker-machine.

3

I have carefully followed those instructions but important details are missing, such as specific software versions and the need to always remove failed hostonlyifs and docker-machines before trying again.

For example, I needed to upgrade my VirtualBox from 4.x to 5.x to make progress past my initial problem I reporte in this thread, because RancherOS ISO failed on the older version.

I still don’t have it working and here are my notes. I would apprecate any help as my company is evaluating RancherOS and Rancher for commercial adoption across our product line but so far, it isn’t looking good.

Versions:

  RancherOS  v0.4.4  (latest stable)

  app-emulation/virtualbox-bin-5.0.20.106931:0
  app-emulation/virtualbox-modules-5.0.20:0

  app-emulation/docker-1.11.0:0          API version 1.23
  app-emulation/docker-machine-0.7.0:0   API version ?.??
  dev-python/docker-py-1.8.0_rc5:0       API version ?.??

  app-emulation/docker-compose-1.7.0:0
  dev-python/dockerpty-0.4.1:0

Steps:

  1. Pre-clear any leftover host interfaces

    $ VBoxManage hostonlyif remove vboxnet0
    $ VBoxManage list hostonlyifs
    (none)

  2. Create the machine, without a hostonly interface

    $ docker-machine create --driver virtualbox --virtualbox-boot2docker-url h ttps://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso Dock1
    Running pre-create checks…
    (Dock1) Boot2Docker URL was explicitly set to h"ttps://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso" at create time, so Docker Machine cannot upgrade this machine to the latest version.
    Creating machine…
    (Dock1) Boot2Docker URL was explicitly set to h"ttps://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso" at create time, so Docker Machine cannot upgrade this machine to the latest version.
    (Dock1) Downloading /home/jrush/.docker/machine/cache/boot2docker.iso from h ttps://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso…
    (Dock1) 0%…10%…20%…30%…40%…50%…60%…70%…80%…90%…100%
    (Dock1) Creating VirtualBox VM…
    (Dock1) Creating SSH key…
    (Dock1) Starting the VM…
    (Dock1) Check network to re-create if needed…
    (Dock1) Found a new host-only adapter: “vboxnet0”
    (Dock1) Waiting for an IP…
    Error creating machine: Error in driver during machine creation: host-only cidr conflicts with the network address of a host interface

    Very odd as I can’t find any such conflict:

    $ ifconfig
    docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
    inet 172.17.0.1 netmask 255.255.0.0 broadcast 0.0.0.0
    ether 02:42:2f:90:3f:f2 txqueuelen 0 (Ethernet)
    RX packets 0 bytes 0 (0.0 B)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 0 bytes 0 (0.0 B)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

    enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
    inet 192.168.2.67 netmask 255.255.255.0 broadcast 192.168.2.255
    inet6 fe80::a9e:1ff:fedf:3c12 prefixlen 64 scopeid 0x20
    ether 08:9e:01:df:3c:12 txqueuelen 1000 (Ethernet)
    RX packets 48773 bytes 68190843 (65.0 MiB)
    RX errors 0 dropped 36 overruns 0 frame 0
    TX packets 22682 bytes 1768144 (1.6 MiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
    device interrupt 16

    lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
    inet 127.0.0.1 netmask 255.0.0.0
    inet6 ::1 prefixlen 128 scopeid 0x10
    loop txqueuelen 1 (Local Loopback)
    RX packets 930 bytes 123342 (120.4 KiB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 930 bytes 123342 (120.4 KiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

    vboxnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
    inet 192.168.99.1 netmask 255.255.255.0 broadcast 192.168.99.255
    inet6 fe80::800:27ff:fe00:0 prefixlen 64 scopeid 0x20
    ether 0a:00:27:00:00:00 txqueuelen 1000 (Ethernet)
    RX packets 0 bytes 0 (0.0 B)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 41 bytes 8795 (8.5 KiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

    wlp2s0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
    inet6 fe80::4ad2:24ff:feae:ad0e prefixlen 64 scopeid 0x20
    ether 48:d2:24:ae:ad:0e txqueuelen 1000 (Ethernet)
    RX packets 41 bytes 5535 (5.4 KiB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 41 bytes 6355 (6.2 KiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

    $ route -n
    Kernel IP routing table
    Destination Gateway Genmask Flags Metric Ref Use Iface
    0.0.0.0 192.168.2.1 0.0.0.0 UG 202 0 0 enp1s0
    172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
    192.168.2.0 0.0.0.0 255.255.255.0 U 202 0 0 enp1s0
    192.168.99.0 0.0.0.0 255.255.255.0 U 207 0 0 vboxnet0

  3. Puzzle: After the run there is an ISO file that I thought would be RancherOS but is tagged as Boot2Docker:

    $ file ~/.docker/machine/cache/boot2docker.iso
    boot2docker.iso: DOS/MBR boot sector ISO 9660 CD-ROM filesystem data ‘Boot2Docker-v1.11.1’ (bootable); partition 1 : ID=0x17, active, start-CHS (0x0,0,1), end-CHS (0x22,63,32), startsector 0, 71680 sectors

  4. Try again, but specify the host-only cidr myself this time

    $ VBoxManage hostonlyif remove vboxnet0
    $ docker-machine rm Dock1
    $ docker-machine create --driver virtualbox --virtualbox-hostonly-cidr “192.168.100.100/24” --virtualbox-boot2docker-url https://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso Dock1
    Running pre-create checks…
    (Dock1) Boot2Docker URL was explicitly set to h"ttps://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso" at create time, so Docker Machine cannot upgrade this machine to the latest version.
    Creating machine…
    (Dock1) Boot2Docker URL was explicitly set to h"ttps://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso" at create time, so Docker Machine cannot upgrade this machine to the latest version.
    (Dock1) Downloading /home/jrush/.docker/machine/cache/boot2docker.iso from https://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso…
    (Dock1) 0%…10%…20%…30%…40%…50%…60%…70%…80%…90%…100%
    (Dock1) Creating VirtualBox VM…
    (Dock1) Creating SSH key…
    (Dock1) Starting the VM…
    (Dock1) Check network to re-create if needed…
    (Dock1) Found a new host-only adapter: “vboxnet0”
    (Dock1) Waiting for an IP…
    Waiting for machine to be running, this may take a few minutes…
    Detecting operating system of created instance…
    Waiting for SSH to be available…
    Detecting the provisioner…
    Provisioning with rancheros…
    Copying certs to the local machine directory…
    Copying certs to the remote machine…
    Setting Docker configuration on the remote daemon…
    Checking connection to Docker…
    Docker is up and running!
    To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env Dock1

  5. Great, I now have a RancherOS machine!

    $ docker-machine env Dock1
    export DOCKER_TLS_VERIFY="1"
    export DOCKER_HOST=“tcp://192.168.100.101:2376"
    export DOCKER_CERT_PATH=”/home/jrush/.docker/machine/machines/Dock1"
    export DOCKER_MACHINE_NAME=“Dock1”

    Run this command to configure your shell:

    eval $(docker-machine env Dock1)

    $ eval $(docker-machine env Dock1)

  6. But the Docker API in Docker and inside RancherOS are different, it seems:

    $ docker ps
    Error response from daemon: client is newer than server (client API version: 1.23, server API version: 1.22)

    $ docker version
    Client:
    Version: 1.11.0
    API version: 1.23
    Go version: go1.6.1
    Git commit: 4dc5990
    Built:
    OS/Arch: linux/amd64
    Error response from daemon: client is newer than server (client API version: 1.23, server API version: 1.22)

    $ docker-machine ip Dock1
    192.168.100.101
    $ docker-machine ssh Dock1

    [docker@Dock1 ~]$ uname -a
    Linux Dock1 4.2.8-ckt4-rancher #1 SMP Tue Apr 5 16:12:00 UTC 2016 x86_64 GNU/Linux

    [docker@Dock1 ~]$ docker version
    Client:
    Version: 1.10.3
    API version: 1.22
    Go version: go1.5.3
    Git commit: 20f81dd
    Built: Thu Mar 10 21:49:11 2016
    OS/Arch: linux/amd64

    Server:
    Version: 1.10.3
    API version: 1.22
    Go version: go1.5.3
    Git commit: 20f81dd
    Built: Thu Mar 10 21:49:11 2016
    OS/Arch: linux/amd64

    So how to update the version of Docker -inside- the RancherOS VM?

  7. Updating version of Docker inside RancherOS

    Back outside of the VM, let’s try the docker-machine update subcommand,
    although there is a warning from RancherOS (in initial docker-machine
    create) that Docker Machine cannot upgrade this machine. But let’s try
    it as it’s the only idea I have.

    Boot2Docker URL was explicitly set to “h ttps://github.com/rancher/os/releases/download/v0.4.4/rancheros.iso”
    _ at create time, so Docker Machine cannot upgrade this machine to the latest version._

    $ docker-machine upgrade Dock1
    Waiting for SSH to be available…
    Detecting the provisioner…
    Upgrading docker…
    Stopping machine to do the upgrade…
    Upgrading machine Dock1…
    Downloading /home/jrush/.docker/machine/cache/boot2docker.iso from h ttps://github.com/rancherio/os/releases/download/v0.3.3/machine-rancheros.iso…
    0%…10%…20%…30%…40%…50%…60%…70%…80%…90%…100%
    Unable to get the local Boot2Docker ISO version: Did not find prefix “-v” in version string
    Default Boot2Docker ISO is out-of-date, downloading the latest release…
    Latest release for github.com/boot2docker/boot2docker is v1.11.1
    Downloading /home/jrush/.docker/machine/cache/boot2docker.iso from h ttps://github.com/boot2docker/boot2docker/releases/download/v1.11.1/boot2docker.iso…
    0%…10%…20%…30%…40%…50%…60%…70%…80%…90%…100%
    Copying /home/jrush/.docker/machine/cache/boot2docker.iso to /home/jrush/.docker/machine/machines/Dock1/boot2docker.iso…
    Starting machine back up…
    (Dock1) Check network to re-create if needed…
    (Dock1) Waiting for an IP…
    Too many retries waiting for SSH to be available. Last error: Maximum number of retries (60) exceeded

    Hmm, that doesn’t look good. It appears it failed to upgrade RancherOS and
    now is -replacing- RancherOS with the latest version of boot2docker.iso.

    $ docker-machine ssh Dock1
    exit status 255

    And now the VM is inaccessible.

I think the core problem now is how to use the latest version of docker client with the latest stable version of RancherOS when their APIs differ and there is no adaptive fallback or upgrade path forward.