Cannot appear to export two ports

[kiboro]

I’m having trouble with a container where I need both http (8080) and https (8443) exporting. I can only seem to export one port.

Both ports are open on the container:
[root@docker-1 ~]# tcping 10.42.68.42 8080
10.42.68.42 port 8080 open.
[root@docker-1 ~]# tcping 10.42.68.42 8443
10.42.68.42 port 8443 open.

But not mapped in CATTLE_PREROUTING:
Chain CATTLE_PREROUTING (1 references)
target prot opt source destination
DNAT tcp – anywhere anywhere ADDRTYPE match dst-type LOCAL tcp dpt:peerwire to:10.42.26.88:9104
DNAT udp – anywhere anywhere ADDRTYPE match dst-type LOCAL udp dpt:ipsec-nat-t to:10.42.145.35:4500
DNAT udp – anywhere anywhere ADDRTYPE match dst-type LOCAL udp dpt:isakmp to:10.42.145.35:500
DNAT tcp – anywhere anywhere ADDRTYPE match dst-type LOCAL tcp dpt:ddi-tcp-1 to:10.42.252.126:80
DNAT tcp – anywhere anywhere ADDRTYPE match dst-type LOCAL tcp dpt:webcache to:10.42.68.42:8080

Needless to say this doesn’t then work from outside Rancher:
[kevin.localhost] → tcping docker-1.pibenchmark.com 8080
Probing 10.20.x.x:8080/tcp - Port is open - time=26.840ms
Probing 10.20.x.x:8080/tcp - Port is open - time=36.188ms

[kevin.localhost] → tcping docker-1.pibenchmark.com 8443
Probing 10.20.x.x:8443/tcp - No response - time=2000.580ms
Probing 10.20.x.x:8443/tcp - No response - time=2000.940ms
Probing 10.20.x.x:8443/tcp - No response - time=2001.085ms
Probing 10.20.x.x:8443/tcp - No response - time=2000.910ms

Should this be possible?

[denise]

Yes, it should be possible.

On the host with the container, could you run the following command and provide the full output?

iptables-save

In my test case I was able to see these rules programmed for my container.

-A CATTLE_PREROUTING -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 8080 -j DNAT --to-destination 10.42.217.29:80
-A CATTLE_PREROUTING -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 8443 -j DNAT --to-destination 10.42.217.29:81

[kiboro]

I gave you the fragment before. Here’s the full thing. As you see there’s no port 8443.

Generated by iptables-save v1.4.21 on Wed Sep 23 07:56:52 2015

*mangle
:PREROUTING ACCEPT [24744597:12168101143]
:INPUT ACCEPT [7223728:7910868197]
:FORWARD ACCEPT [16818959:4110328696]
:OUTPUT ACCEPT [5625570:5101398068]
:POSTROUTING ACCEPT [22444227:9211605220]
COMMIT

Completed on Wed Sep 23 07:56:52 2015

Generated by iptables-save v1.4.21 on Wed Sep 23 07:56:52 2015

*nat
:PREROUTING ACCEPT [265:34395]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [3:189]
:POSTROUTING ACCEPT [13:789]
:CATTLE_POSTROUTING - [0:0]
:CATTLE_PREROUTING - [0:0]
:DOCKER - [0:0]
-A PREROUTING -j CATTLE_PREROUTING
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -j CATTLE_POSTROUTING
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.17.0.95/32 -d 172.17.0.95/32 -p udp -m udp --dport 4500 -j MASQUERADE
-A POSTROUTING -s 172.17.0.95/32 -d 172.17.0.95/32 -p udp -m udp --dport 500 -j MASQUERADE
-A CATTLE_POSTROUTING -s 10.42.0.0/16 -d 169.254.169.250/32 -j ACCEPT
-A CATTLE_POSTROUTING -s 10.42.0.0/16 ! -d 10.42.0.0/16 -p tcp -j MASQUERADE --to-ports 1024-65535
-A CATTLE_POSTROUTING -s 10.42.0.0/16 ! -d 10.42.0.0/16 -p udp -j MASQUERADE --to-ports 1024-65535
-A CATTLE_POSTROUTING -s 10.42.0.0/16 ! -d 10.42.0.0/16 -j MASQUERADE
-A CATTLE_POSTROUTING -s 172.17.0.0/16 ! -o docker0 -p tcp -j MASQUERADE --to-ports 1024-65535
-A CATTLE_POSTROUTING -s 172.17.0.0/16 ! -o docker0 -p udp -j MASQUERADE --to-ports 1024-65535
-A CATTLE_PREROUTING -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 9104 -j DNAT --to-destination 10.42.26.88:9104
-A CATTLE_PREROUTING -p udp -m addrtype --dst-type LOCAL -m udp --dport 4500 -j DNAT --to-destination 10.42.145.35:4500
-A CATTLE_PREROUTING -p udp -m addrtype --dst-type LOCAL -m udp --dport 500 -j DNAT --to-destination 10.42.145.35:500
-A CATTLE_PREROUTING -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 8888 -j DNAT --to-destination 10.42.252.126:80
-A CATTLE_PREROUTING -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 8080 -j DNAT --to-destination 10.42.146.53:8080
-A DOCKER ! -i docker0 -p udp -m udp --dport 4500 -j DNAT --to-destination 172.17.0.95:4500
-A DOCKER ! -i docker0 -p udp -m udp --dport 500 -j DNAT --to-destination 172.17.0.95:500
COMMIT

Completed on Wed Sep 23 07:56:52 2015

Generated by iptables-save v1.4.21 on Wed Sep 23 07:56:52 2015

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [1243:76908]
:OUTPUT ACCEPT [3655420:3483738063]
:DOCKER - [0:0]
:f2b-SSH - [0:0]
:fail2ban-SSH - [0:0]
-A INPUT -p tcp -m tcp --dport 22 -j f2b-SSH
-A INPUT -i lo -m comment --comment “000 accept all to lo interface” -j ACCEPT
-A INPUT -p icmp -m comment --comment “001 accept all icmp” -j ACCEPT
-A INPUT -m comment --comment “002 accept related established rules” -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m multiport --ports 22 -m comment --comment “099 send ssh connections to fail2ban chain” -m state --state NEW -j fail2ban-SSH
-A INPUT -s 10.20.0.0/16 -p tcp -m multiport --ports 22 -m comment --comment “100 accept ssh for PI” -j ACCEPT
-A INPUT -s 10.20.0.0/16 -p tcp -m multiport --ports 5666 -m comment --comment “110 accept nrpe for nagios” -j ACCEPT
-A INPUT -s 10.20.0.0/16 -p tcp -m multiport --ports 2375 -m comment --comment “200 docker for PI” -j ACCEPT
-A INPUT -s 10.20.0.0/16 -p tcp -m multiport --ports 5555 -m comment --comment “200 pi_sysadmin accept for PI” -j ACCEPT
-A INPUT -m comment --comment “999 drop all” -j DROP
-A FORWARD -s 10.20.0.0/16 -p tcp -m multiport --ports 8000:9999 -m comment --comment “200 docker forwards for PI” -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.17.0.95/32 ! -i docker0 -o docker0 -p udp -m udp --dport 4500 -j ACCEPT
-A DOCKER -d 172.17.0.95/32 ! -i docker0 -o docker0 -p udp -m udp --dport 500 -j ACCEPT
-A f2b-SSH -j RETURN
-A fail2ban-SSH -p tcp -m comment --comment “099 return from fail2ban chain” -j RETURN
COMMIT

Completed on Wed Sep 23 07:56:52 2015

[kiboro]

This is the docker-compose file

keycloak-140-220915-01:
image: private-registry.com/pi-keycloak:1.4.0-220915-01
environment:
KEYSTORE: keystore.jks
KEYSTORE_PASSWORD: XXXXX
MYSQL_SERVER: mysql
MYSQL_USER: keycloak
MYSQL_PASSWORD: XXXXX
ports:
- “8080:8080/tcp”
- "8443:8443/tcp"
links:
- keycloak-db:mysql
tty: true
stdin_open: true
keycloak-db:
image: mysql/mysql-server:5.6
environment:
MYSQL_ROOT_PASSWORD: XXXXX
MYSQL_DATABASE: keycloak
MYSQL_USER: keycloak
MYSQL_PASSWORD: XXXXX
volumes_from:
- keycloak-data
labels:
io.rancher.sidekicks: keycloak-data
tty: true
stdin_open: true
keycloak-data:
image: mysql/mysql-server:5.6
command:
- cat
tty: true

[kiboro]

Interestingly this IS only one port allowed it seems.

ports:
- “8443:8443/tcp”
- “8001:8001/tcp”
- “9001:9001/tcp”
- “8080:8080/tcp”

and only one of the four ports appears in iptables

-A CATTLE_PREROUTING -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 9104 -j DNAT --to-destination 10.42.26.88:9104
-A CATTLE_PREROUTING -p udp -m addrtype --dst-type LOCAL -m udp --dport 4500 -j DNAT --to-destination 10.42.145.35:4500
-A CATTLE_PREROUTING -p udp -m addrtype --dst-type LOCAL -m udp --dport 500 -j DNAT --to-destination 10.42.145.35:500
-A CATTLE_PREROUTING -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 8888 -j DNAT --to-destination 10.42.252.126:80
-A CATTLE_PREROUTING -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 8001 -j DNAT --to-destination 10.42.244.44:8001

[denise]

This seems to match this open issue in Rancher.

https://github.com/rancher/rancher/issues/1898