Rancher Not Using Internal Address for Container Traffic

I’m trying to figure out how to make Rancher use a specific subnet for all traffic that doesn’t need to accessible outside of K8s cluster. The servers my cluster is running on has 3 networks a for management of the physical hosts, for external docker traffic and a for internal docker traffic. The internal network is 10gb and external network is 1gb so I want all the internal traffic using the faster network. This is especially important for stuff like Longhorn that needs to move a lot of data.

My nodes are just docker on bare metal started with a command like this and when I do a tcpdump against the 1gb interface I see all of traffic there.

Note that rancher-private resolves to a address and I do see traffic on port 6443 between the rancher agents on the 10gb network but nothing else

sudo docker run -d --privileged --restart=unless-stopped --net=host -v /etc/kubernetes:/etc/kubernetes -v /var/run:/var/run rancher/rancher-agent:v2.3.2 --server https://rancher-private.dev.example.com --token zf4bkvvjkn4q5547gkgc6x8bd5nnl47zthl6t5lmthv7gs4h5q6qzz --ca-checksum 75e28964c7f30bfbb2e3e30e458b557c3d6197664159767356b486a428893c00 --address --internal-address --worker

1 Like

Pretty much same issue here.

When using the internal address and address, it should route properly. This sounds like a bug. If you can confirm the behavior, please put an issue in. https://github.com/rancher/rancher/issues/new


Nr. 1 is the IP address that is desired and that is set in the command starting the worker node, as shown below.
Nr. 2 is the other IP address I do not want to be part of Rancher at all. The necessary setup for the network traffic is done outside of Rancher. All Rancher needs to do is using the first desired IP address, which is in the below example. It should ignore the other address entirely and it should not show up in any YAMLs or anywhere within Rancher (which it does).

Starting worker node with:

sudo docker run -d --privileged --restart=unless-stopped --net=host \
-v /etc/kubernetes:/etc/kubernetes \
-v /var/run:/var/run rancher/rancher-agent:v2.3.3 \
--server --address --internal-address \
--token theactualtoken --ca-checksum thcachecksum --worker