Manipulating mangle table in prerouting to secure Rancher 1.6 host


#1

I’m trying to secure a rancher host by creating some restrictive rules in the mangle table on the prerouting chain. That is to say, I want to restrict it to the basics (ssh, icmp, http/s and whatever the rancher server needs access to).
The ip of the machine is: 10.0.2.15 (on enp0s3 - the interface facing the internet)
Rancher server (which is running on port 8080) and rancher host are on another network (192.168.99.0/24) and the host’s server is 192.168.99.22 and the server’s ip is 192.168.99.44.
DNS of the host is: 10.0.2.3 (internally, of course - systemd-resolved or dnsmasq or whatever that is).
I know restricting access like that in this table is against any common-sense, but as it is I’ve got no other choice.
These are the current rules that I’ve applied in the mangle table:

-A PREROUTING -i docker0 -j ACCEPT
-A PREROUTING -s 10.0.2.3/32 -p udp -m udp --sport 53 -j ACCEPT
-A PREROUTING -s 10.0.2.3/32 -p tcp -m tcp --sport 53 -j ACCEPT
-A PREROUTING -p icmp -j ACCEPT
-A PREROUTING -s 192.168.99.44/32 -j LOG --log-prefix "RANCHERSERVER: " --log-level 6
-A PREROUTING -s 192.168.99.44/32 -p tcp -m multiport --dports 32000:65535 -j ACCEPT
-A PREROUTING -s 192.168.99.44/32 -p udp -m multiport --dports 4500,500 -j ACCEPT
-A PREROUTING -p tcp -m multiport --dports 8080,443,80 -j ACCEPT
-A PREROUTING -p tcp -m tcp --dport 22 -j ACCEPT
-A PREROUTING -j DROP

For some reason, when I create a stack of containers with docker-compose (in my case mattermost), it doesn’t work. It keeps saying “waiting”. If I delete the last rule (DROP), it works again. I’m not sure what type of traffic I’m supposed to allow, but any help would be much appreciated.
I’m running Ubuntu 16.04 on both with docker 18.06.3-ce


#2

I ended up with these rules:

-A PREROUTING -i enp0s3 -p tcp -m multiport --dports 32000:65535
-A PREROUTING -i enp0s3 -p udp -m multiport --dports 32000:65535
-A PREROUTING -i docker0 -j ACCEPT
-A PREROUTING -s 10.0.2.3/32 -p tcp -m tcp --sport 53 -j ACCEPT
-A PREROUTING -s 10.0.2.3/32 -p udp -m udp --sport 53 -j ACCEPT
-A PREROUTING -s 1.1.1.1/32 -p udp -m udp --sport 53 -j ACCEPT
-A PREROUTING -s 1.1.1.1/32 -p tcp -m tcp --sport 53 -j ACCEPT
-A PREROUTING -p icmp -j ACCEPT
-A PREROUTING -p tcp -m multiport --sports 8080,443,80 -j ACCEPT
-A PREROUTING -s 192.168.99.44/32 -p tcp -m multiport --dports 32000:65535 -j ACCEPT
-A PREROUTING -s 192.168.99.44/32 -p udp -m multiport --dports 4500,500 -j ACCEPT
-A PREROUTING -p tcp -m multiport --dports 8080,443,80 -j ACCEPT
-A PREROUTING -p tcp -m tcp --dport 22 -j ACCEPT
-A PREROUTING -m limit --limit 1/sec -j LOG --log-prefix "DROPPED: "
-A PREROUTING -j DROP

I allowed access to non-privileged ports on my host (started with port 32000, which I guess is not canonical, but should cover all ports used by the client which initiates connections).

I’ve also allowed access from http/s connections (in my case github packages were initially dropped - that’s why the mattermost stack didn’t work), as in this case you need to think that this also affects sessions started from your own host. What is initially self-understood (return packages of the sessions originating from your host are allowed) needs to become explicit in the PREROUTING.
Really nasty things I (and others) am forced into, unfortunately.


#3

A much simpler way based on conntrack as first rule:

-A PREROUTING -i eth0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PREROUTING -i docker0 -j ACCEPT
-A PREROUTING -s rancher_server_ip/32 -p tcp -m multiport --dports 32000:65535 -j ACCEPT
-A PREROUTING -s rancher_server_ip/32 -p udp -m multiport --dports 4500,500 -j ACCEPT
-A PREROUTING -p tcp -m multiport --dports 8080,8000,443,80 -j ACCEPT
-A PREROUTING -p tcp -m tcp --dport 22 -j ACCEPT
-A PREROUTING -m limit --limit 1/min -j LOG --log-prefix "DROPPED: "
-A PREROUTING -j DROP