Cannot add hosts. 'Connection aborted.', error(111, 'Connection refused')

Hi,

It seems that I cannot add any hosts using Custom option. This is the setup:

I have a host called andonmaster and it’s running the rancher. Then I have one host named dockerregistry which I want to add to rancher. But this simple job seems to be too hard for me or for our env…

We have our own cloud where we must use domain names. Is that somehow the factor? IMO the iptables are ok in both hosts…

@andonmaster:~$ sudo iptables -L
[sudo] password for makelkal:
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER all – anywhere anywhere
ACCEPT all – anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all – anywhere anywhere
ACCEPT all – anywhere anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp – anywhere 172.17.0.1 tcp dpt:http-alt
ACCEPT tcp – anywhere 172.17.0.25 tcp dpt:9200
ACCEPT tcp – anywhere 172.17.0.26 tcp dpt:5000
ACCEPT tcp – anywhere 172.17.0.27 tcp dpt:5601
ACCEPT tcp – anywhere 172.17.0.182 tcp dpt:http-alt
ACCEPT tcp – anywhere 172.17.1.208 tcp dpt:http-alt

@dockerregistry:~$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp – anywhere anywhere tcp dpt:ssh state NEW,ESTABLISHED
ACCEPT tcp – anywhere anywhere tcp dpt:http state NEW,ESTABLISHED
ACCEPT tcp – anywhere anywhere tcp dpt:https state NEW,ESTABLISHED
ACCEPT icmp – anywhere anywhere icmp echo-request
ACCEPT icmp – anywhere anywhere icmp echo-reply
ACCEPT udp – anywhere anywhere udp spt:domain

Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER all – anywhere anywhere
ACCEPT all – anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all – anywhere anywhere
ACCEPT all – anywhere anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp – anywhere anywhere tcp spt:ssh state ESTABLISHED
ACCEPT tcp – anywhere anywhere tcp spt:http state ESTABLISHED
ACCEPT tcp – anywhere anywhere tcp spt:https state ESTABLISHED
ACCEPT icmp – anywhere anywhere icmp echo-reply
ACCEPT icmp – anywhere anywhere icmp echo-request
ACCEPT udp – anywhere anywhere udp dpt:domain

Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp – anywhere 172.17.0.1 tcp dpt:5000
ACCEPT tcp – anywhere 172.17.0.6 tcp dpt:tproxy
ACCEPT tcp – anywhere 172.17.0.29 tcp dpt:http-alt
ACCEPT tcp – anywhere 172.17.0.63 tcp dpt:http-alt
ACCEPT tcp – anywhere 172.17.0.88 tcp dpt:http-alt
ACCEPT tcp – anywhere 172.17.0.115 tcp dpt:http-alt
ACCEPT tcp – anywhere 172.17.0.122 tcp dpt:http-alt

@dockerregistry:~$ docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock rancher/agent:v0.8.2 (could not use http here because I’m allowd only two links :smile: )://andonmaster.andonsandbox.xxxxxxx.com:8090/v1/scripts/{KEY}
4d1f7897e7b98002fdabd3ef420ed13784142d97908ebd0f6cf3935091dd3e57

@dockerregistry:~$ docker logs 4d1f7897e7b9

INFO: Running Agent Registration Process, CATTLE_URL=http://andonmaster.andonsandbox.pdcloud.xxxxx.com:8090/v1
INFO: Checking for Docker version >= 1.6.0
INFO: Found Server version: 1.8.1
INFO: docker version: Client version: 1.6.0
INFO: docker version: Client API version: 1.18
INFO: docker version: Go version (client): go1.4.2
INFO: docker version: Git commit (client): 4749651
INFO: docker version: OS/Arch (client): linux/amd64
INFO: docker version: Server version: 1.8.1
INFO: docker version: Server API version: 1.20
INFO: docker version: Go version (server): go1.4.2
INFO: docker version: Git commit (server): d12ea79
INFO: docker version: OS/Arch (server): linux/amd64
INFO: docker info: Containers: 22
INFO: docker info: Images: 261
INFO: docker info: Storage Driver: aufs
INFO: docker info: Root Dir: /var/lib/docker/aufs
INFO: docker info: Backing Filesystem: extfs
INFO: docker info: Dirs: 306
INFO: docker info: Dirperm1 Supported: false
INFO: docker info: Execution Driver: native-0.2
INFO: docker info: Kernel Version: 3.13.0-62-generic
INFO: docker info: Operating System: Ubuntu 14.04.3 LTS
INFO: docker info: CPUs: 1
INFO: docker info: Total Memory: 7.798 GiB
INFO: docker info: Name: dockerregistry
INFO: docker info: ID: ETJJ:4A6M:VBZQ:4OEI:WZ3T:SJCL:MGJ6:7VRJ:6MNG:BO7M:2T5D:JOTO
INFO: docker info: Http Proxy:
INFO: docker info: Https Proxy:
INFO: docker info: No Proxy:
WARNING: No swap limit support
INFO: Attempting to connect to: http://andonmaster.andonsandbox.xxxxx:8090/v1
INFO: http://andonmaster.andonsandbox.xxxxx:8090/v1 is accessible
INFO: Inspecting host capabilities
INFO: System: false
INFO: Host writable: true
INFO: Token: xxxxxxxx
INFO: Running registration
Traceback (most recent call last):
File “./register.py”, line 11, in
secret_key=os.environ[‘CATTLE_REGISTRATION_SECRET_KEY’])
File “/usr/local/lib/python2.7/dist-packages/cattle.py”, line 37, in from_env
return gdapi.from_env(prefix=prefix, factory=Client, **kw)
File “/usr/local/lib/python2.7/dist-packages/gdapi.py”, line 608, in from_env
return _from_env(prefix=prefix, factory=factory, **args)
File “/usr/local/lib/python2.7/dist-packages/gdapi.py”, line 627, in _from_env
return factory(**result)
File “/usr/local/lib/python2.7/dist-packages/cattle.py”, line 12, in init
super(Client, self).init(*args, **kw)
File “/usr/local/lib/python2.7/dist-packages/gdapi.py”, line 195, in init
self._load_schemas()
File “/usr/local/lib/python2.7/dist-packages/gdapi.py”, line 313, in _load_schemas
response = self._get_response(self._url)
File “/usr/local/lib/python2.7/dist-packages/gdapi.py”, line 262, in _get_response
headers=self._headers)
File “/usr/local/lib/python2.7/dist-packages/requests/sessions.py”, line 477, in get
return self.request(‘GET’, url, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/requests/sessions.py”, line 465, in request
resp = self.send(prep, **send_kwargs)
File “/usr/local/lib/python2.7/dist-packages/requests/sessions.py”, line 573, in send
r = adapter.send(request, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/requests/adapters.py”, line 415, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: (‘Connection aborted.’, error(111, ‘Connection refused’))

Thanks!

  • Kalle

This looks a lot like this issue: https://github.com/rancher/rancher/issues/1147

Basically, if you have some containers on your host that were deployed using Docker 1.5, the containers are causing issues when deploying Rancher on a later version of Rancher. You can fix this by stopping and removing the containers deployed using Docker 1.5.

Basically, the concept of labels on Docker images/containers changed, which caused issues.

Thanks Denise for answering, but I’m quite sure that Docker 1.7 is the oldest version that these hosts have ever had installed.

Now I have:

@andonmaster:~$ docker version
Client:
Version: 1.8.1
API version: 1.20
Go version: go1.4.2
Git commit: d12ea79
Built: Thu Aug 13 02:35:49 UTC 2015
OS/Arch: linux/amd64

Server:
Version: 1.8.1
API version: 1.20
Go version: go1.4.2
Git commit: d12ea79
Built: Thu Aug 13 02:35:49 UTC 2015
OS/Arch: linux/amd64

@dockerregistry:~$ docker version
Client:
Version: 1.8.1
API version: 1.20
Go version: go1.4.2
Git commit: d12ea79
Built: Thu Aug 13 02:35:49 UTC 2015
OS/Arch: linux/amd64

Server:
Version: 1.8.1
API version: 1.20
Go version: go1.4.2
Git commit: d12ea79
Built: Thu Aug 13 02:35:49 UTC 2015
OS/Arch: linux/amd64

Hi,

I got it to work. The reason was that my iptables was not reloaded… Sorry for inconvenience!

Lessons learnt: If the error says that connection is refused, it usually is :smile:

Thanks!
Kalle