No connection to linked services on other hosts

Archive Rancher 1.x
1

Since I upgraded to v0.50.0 (issue still happens in v0.51.0) I cannot access any linked services which reside on different hosts.

The hostname and dns are all mapped and resolve correctly, however I am unable to connect across hosts using the internal (10.x) ip address.

From another container running ont he same host this works correctly.

I have set this up for many services before where i used service links and connected tot hem by link name regardless of which host they live on.

Not sure what I am doing wrong here.

2

It sounds like your cross host communication between the hosts have failed. Are you able to exec into the network agent and ping the other IP address of the network agent on the other host?

Can you also make sure that docker is still okay on the hosts? Log in to the host and try a docker ps.

3

docker is running correctly on both hosts. I was unable to exec intot he network container through the rancher ui, but it did work froma terminal.

matt@–:~$ docker exec -it a1da1cee5107 bash
root@a1da1cee5107:/# ping 10.42.197.23
PING 10.42.197.23 (10.42.197.23) 56(84) bytes of data.
From 10.42.1.36 icmp_seq=1 Destination Host Unreachable
From 10.42.1.36 icmp_seq=2 Destination Host Unreachable
From 10.42.1.36 icmp_seq=3 Destination Host Unreachable
From 10.42.1.36 icmp_seq=4 Destination Host Unreachable
From 10.42.1.36 icmp_seq=5 Destination Host Unreachable
From 10.42.1.36 icmp_seq=6 Destination Host Unreachable
^C
— 10.42.197.23 ping statistics —
8 packets transmitted, 0 received, +6 errors, 100% packet loss, time 7039ms
pipe 3

4

Sounds like this issue:

5

Definitely seems similar, though I am running only 2 hosts and 2 containers. Laso I am not seeing it working ever, that being said im not monitoring constantly

6

@Matt_Rochon,

I was seeing it with hosts that were just brought up with only the networking container running. It was going on for about 30 minutes before they all able to ping each other.

I do know that they are working upgrading the ipsec software that is used for the managed networking.

7

I started my hosts last night, still unable to connect cross-host, though the correct ips do resolve with the link/host name

8

if you only have 1 container on each host, you may be hitting a different issue, where sometimes pinging between containers doesn’t work if you only have 1 container and you need to launch another container on the host to get it working.

https://github.com/rancher/rancher/issues/486