Hello.
Need some help with Docker network config. By default my containers don’t have any network\internet access. My host machine has two interface (one(5.61.0.0) to local network, and another one to internet(10.38.0.0))
The only solution i found is to add iptable rule:
iptables -t nat -I POSTROUTING -p all -s 172.17.0.0/16 -j SNAT --to-source <my-host-ip-from-5.61.0.0-network>
But now if use Rancher default overlay network (10.42.0.0), containers can’t communicate with each other on different hosts by using 10.42.X.X ip. How can i configure Docker networking and\or Rancher to make it work together?
My system and network info:
-----
docker info
Containers: 35
Running: 32
Paused: 0
Stopped: 3
Images: 4
Server Version: 1.11.2
Storage Driver: overlay
Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge null host
Kernel Version: 4.6.2-1.el7.elrepo.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 24
Total Memory: 31.39 GiB
Name: host.dev.dv
ID: W25R:CFB7:J3E7:SGDY:YEBF:BWHR:BR2F:LAJZ:U5FF:ONLM:ZXBQ:ZASQ
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
-----------------
# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
CATTLE_PREROUTING all -- anywhere anywhere
DOCKER all -- anywhere anywhere ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
DOCKER all -- anywhere !loopback/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
SNAT all -- 172.17.0.0/16 anywhere to:5.61.234.27
CATTLE_POSTROUTING all -- anywhere anywhere
MASQUERADE all -- 172.17.0.0/16 anywhere
MASQUERADE udp -- 172.17.0.2 172.17.0.2 udp dpt:ipsec-nat-t
MASQUERADE udp -- 172.17.0.2 172.17.0.2 udp dpt:isakmp
Chain CATTLE_POSTROUTING (1 references)
target prot opt source destination
ACCEPT all -- 10.42.0.0/16 169.254.169.250
MASQUERADE tcp -- 10.42.0.0/16 !10.42.0.0/16 masq ports: 1024-65535
MASQUERADE udp -- 10.42.0.0/16 !10.42.0.0/16 masq ports: 1024-65535
MASQUERADE all -- 10.42.0.0/16 !10.42.0.0/16
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x3ad7e to:10.42.241.22
MASQUERADE tcp -- 172.17.0.0/16 anywhere masq ports: 1024-65535
MASQUERADE udp -- 172.17.0.0/16 anywhere masq ports: 1024-65535
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
SNAT all -- !10.42.0.0/16 169.254.169.250 mark match 0x34016 to:10.42.213.14
Chain CATTLE_PREROUTING (1 references)
target prot opt source destination
DNAT tcp -- 10.42.0.0/16 10.42.0.1 tcp dpt:domain to:169.254.169.250
DNAT udp -- 10.42.0.0/16 10.42.0.1 udp dpt:domain to:169.254.169.250
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:DC:60:F3 MARK set 0x3ad7e
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
MARK all -- !10.42.0.0/16 169.254.169.250 MAC 02:69:63:96:CE:07 MARK set 0x34016
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere
DNAT udp -- anywhere anywhere udp dpt:ipsec-nat-t to:172.17.0.2:4500
DNAT udp -- anywhere anywhere udp dpt:isakmp to:172.17.0.2:500
----------------
# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.38.158.1 0.0.0.0 UG 0 0 0 eth0.3003
10.38.158.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0.3003
10.42.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
10.255.2.0 10.38.158.1 255.255.255.0 UG 0 0 0 eth0.3003
link-local 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
link-local 0.0.0.0 255.255.0.0 U 1029 0 0 extif0
link-local 0.0.0.0 255.255.0.0 U 1041 0 0 eth0.3003
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
--------------
# ifconfig
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 0.0.0.0
inet6 fe80::42:97ff:fe54:899a prefixlen 64 scopeid 0x20<link>
ether 02:22:97:55:89:9a txqueuelen 0 (Ethernet)
RX packets 18078007 bytes 10782154070 (10.0 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15374194 bytes 23394143099 (21.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::67d:7bff:fef1:a94c prefixlen 64 scopeid 0x20<link>
ether 09:7d:bb:f1:a9:4c txqueuelen 1000 (Ethernet)
RX packets 27472778 bytes 26086394861 (24.2 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 23364882 bytes 11529001372 (10.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xdfd20000-dfd3ffff
eth0.3003: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.38.158.34 netmask 255.255.254.0 broadcast 10.38.159.255
inet6 fe80::67d:7bff:fef1:a94c prefixlen 64 scopeid 0x20<link>
ether 09:7d:bb:f1:a9:4c txqueuelen 1000 (Ethernet)
RX packets 16103286 bytes 25113755309 (23.3 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 18688617 bytes 11220213569 (10.4 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
extif0: flags=195<UP,BROADCAST,RUNNING,NOARP> mtu 1500
inet 5.61.234.27 netmask 255.255.255.255 broadcast 5.61.234.27
inet6 fe80::a022:eff:fe5e:cf3a prefixlen 64 scopeid 0x20<link>
ether a5:22:0d:5e:cf:3a txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3 bytes 210 (210.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 74882 bytes 1025810342 (978.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 74882 bytes 1025810342 (978.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vethc57e0d2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::ec92:35ff:fe47:aa44 prefixlen 64 scopeid 0x20<link>
ether ee:92:35:47:aa:44 txqueuelen 0 (Ethernet)
RX packets 1144 bytes 95187 (92.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1084 bytes 12703905 (12.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255
ether 52:54:00:5e:40:52 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
----