Add a new Rancher Agent node, can't find it after join it to cluster

Rancher 1.x
1

I have created 1 Rancher Server master and 2 Rancher Agent nodes. Now tried to add a third one to the cluster. Trouble happened:

[vagrant@node3 ~]$ docker ps
CONTAINER ID        IMAGE                  COMMAND             CREATED             STATUS                  PORTS               NAMES
d87ff3c8ff68        rancher/agent:v1.2.6   "/run.sh run"       4 seconds ago       Up Less than a second                       rancher-agent
[vagrant@node3 ~]$ docker logs d87f
Found container ID: 42f7675fd60e2be87871937a64cf58bce7aaac5b71394c5880bb2ca6dff3654b
Checking root: /host/run/runc
Checking root: /host/var/run/runc
Checking root: /host/run/docker/execdriver/native
Checking root: /host/var/run/docker/execdriver/native
time="2017-11-22T05:18:03Z" level=fatal msg="Failed to find state.json"
Found container ID: fb59b00d89728a2f26761c32f305886131c8be299b8d461cc2df83d3a0e87077
Checking root: /host/run/runc
Checking root: /host/var/run/runc
Checking root: /host/run/docker/execdriver/native
Checking root: /host/var/run/docker/execdriver/native
time="2017-11-22T05:18:03Z" level=fatal msg="Failed to find state.json"
Found container ID: 60e6639f3dd0859458a3b282f27110e9c6e858c6773f2ea6260602e11ffab0c1
Checking root: /host/run/runc
Checking root: /host/var/run/runc
Checking root: /host/run/docker/execdriver/native
Checking root: /host/var/run/docker/execdriver/native
time="2017-11-22T05:18:04Z" level=fatal msg="Failed to find state.json"
Found container ID: 5fe85afde62a25b478597df76e77e6e96429cb24c44e89e44debbd89d79d8a6b
Checking root: /host/run/runc
Checking root: /host/var/run/runc
Checking root: /host/run/docker/execdriver/native
Checking root: /host/var/run/docker/execdriver/native
time="2017-11-22T05:18:06Z" level=fatal msg="Failed to find state.json"
Found container ID: d7b338c5ab11f9e68673973845fb1b4c15f1b929e7ec3f93334fd809bdddeeb7
Checking root: /host/run/runc
Checking root: /host/var/run/runc
Checking root: /host/run/docker/execdriver/native
Checking root: /host/var/run/docker/execdriver/native
time="2017-11-22T05:18:08Z" level=fatal msg="Failed to find state.json"
Found container ID: 32d6d09c65e16c4a74b11dcead93191e1dccfaef211fd5e4ce75739390631a36
Checking root: /host/run/runc
Checking root: /host/var/run/runc
Checking root: /host/run/docker/execdriver/native
Checking root: /host/var/run/docker/execdriver/native
time="2017-11-22T05:18:10Z" level=fatal msg="Failed to find state.json"

Why it Failed to find state.json? What kind of file it is?

2

Maybe because of the version of docker.

Docker version on old Rancher Agent nodes:

[vagrant@node2 ~]$ docker version
Client:
 Version:      17.10.0-ce
 API version:  1.33
 Go version:   go1.8.3
 Git commit:   f4ffd25
 Built:        Tue Oct 17 19:04:05 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.10.0-ce
 API version:  1.33 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   f4ffd25
 Built:        Tue Oct 17 19:05:38 2017
 OS/Arch:      linux/amd64
 Experimental: false

Docker version in new host:

[vagrant@node3 ~]$ docker version
Client:
 Version:      17.11.0-ce
 API version:  1.34
 Go version:   go1.8.3
 Git commit:   1caf76c
 Built:        Mon Nov 20 18:35:47 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.11.0-ce
 API version:  1.34 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   1caf76c
 Built:        Mon Nov 20 18:45:06 2017
 OS/Arch:      linux/amd64
 Experimental: false
3

After uninstall the newest docker version and installed an old version. It works.

Install Docker Engine on CentOS | Docker Docs

How can I install a specific version of the docker engine - DockerEngine - Docker Community Forums

4

According to Rancher’s Supported Docker Versions doc, Docker 17.10.x and 17.11.x aren’t supported. Docker 17.12.x is supported since Rancher 1.6.14, though.

5

Though we don’t test/support docker edge versions, 17.10 and 17.11 would probably work with Rancher 1.6.14+. That error is the same reason 17.12 is not supported until that version, docker moved the state file.