Private LAN hosts + Public WAN hosts on the same mesh? Is there anything I can do?

Rancher 1.x
1

image.png1215Ă—748 75.6 KB

Greetings. I took up Docker and RancherOS/Rancher last Thursday (literally). Been enthralled ever since! :slight_smile:

Anyways, I think I understand how the SDN meshes are established, but was curious if anyone could comment on if there was any way of getting all 4 of these hosts to inter-communicate on the SDN overlayer? For discussion, I will call them Host 1 - 4 (ordered as shown). Hosts 1 and 3 are on an internal Xenserver host. Hosts 2 and 4 are on Linode and Vultr respectively. 1 & 3 can inter-communicate, as can 2 and 4. But since 1 & 3 are on internal private subnet, they cannot intercom with 2 & 4. And, I understand why.

My question is, is there anything that can be done about this? I’m testing on my residential cable Internet (DHCP assigned). My firewall is pfsense. I already have 2 site-to-site IPSEC tunnels connected to this pfsense for work. I have no static IPs for my home Internet service. All hosts were custom adds. I assume if I had static IPs (to spare) I could 1:1 NAT them to each internal host. But shy of that, is there any other way to get all 4 on the same mesh? Since I have the site-to-site tunnels, don’t think I can port forward/pass-through any IPSEC. I’ve read that using only IPV6 could workaround this, but not even sure my residential cable internet connection can or will route IPV6 (I’ve never looked into it).

Anyways, would very much appreciate some more expert insights as I’m finally trying to wrestle a thorough understanding of the container-way.

Thanks!

Mark

2

In short: If the hosts are not reachable via their IP addresses, then the communication is broken.

When you say you have site to site IPSec tunnels already, I am assuming the IP addresses are reachable over these tunnels.

Experiment 1:
If you already have IPSec tunnels, you could try using the overlay IP addresses when registering these hosts in Cattle. But this would mean double IPSec encapsulation. IPSec does NAT traversal but not sure if it works with double IPSec Encapsulation.

Experiment 2:
You could disable IPSec and use VXLAN. Register the hosts using your IPSec IP addresses.

You need to specify the IP address here:

add_host_with_ip.png5359Ă—3435 1.1 MB

3

The existing tunnels are mutually exclusive to my RancherOS/Rancher exploration. All I was trying to do is see if the mesh network would transparently extend between multiple internel, private LAN IP hosts, and multiple external Public WAN IP hosts, with pfsense in the middle running pure NAT, but not static outbound ports. I knew the two private hosts could see each other (by their host IPs), and that the two public hosts could see each other (no iptables in effect).

But what I had hoped for was that even though the public hosts couldn’t see the private hosts, that private hosts could see the public hosts (which they can ping), and that perhaps a tunnel in that direction (via NAT and/or IPSEC passthrough) would suffice. I even setup a static outbound NAT for UDP 500 and 4500 per one individual’s suggest in another forum. No go though.

Which does make we curious. Is there a way in Rancher (or RancherOS or ???) to view what tunnels the management network has setup/active? As an IPSEC passthrough setup can, in theory, permit a tunnel to be established from a private LAN host behind NAT to an Public IP gateway/endpoint, Perhaps the tunnels from private LAN to public WAN hosts are being established, and this is might be a Phase 2 issue?

Interesting idea, too, Leo, to tunnel the hosts first then trying registering them “internally”. I’ve messed around with multi-P2 layers with multiple remote subnets with some success. It would be a good experiment to see if such a nested tunnel could work. Pretty? No, but very cool if it could! :slight_smile: And, I’ve seen reference to VXLAN but not thought to explore it yet, So, will look at that too. Thanks!

My biggest reason for trying to see if such a scenario could be made to work (near) transparently is that I can see where a client that hasn’t yet gone “cloud” except for maybe backups and storage (and perhaps Office 365 or Google Apps), might now have more creative ways of integrating hybrid cloud technologies into an existing infrastructure through “affordable” orchestration offered by RancherLabs, Docker, Kubernetes, etc., that can be deployed without requiring silos or farms or new financing :slight_smile: (at least not required up front). A lot of clients I work with have your typical on-premise resources (NAT firewall, Internet). So, for this to work as I’m exploring it out-of-the-box would, I think, be a catalyst for them to consider broader cloud adoption, and, new avenues of work for me.

Accessible and FOSS cloud orchestration choices through containers may end up doing for cloud technologies what Linux+FOSS+choices did for IT! Though I may seem late for the party, I’ve long been standing outside waiting for my invitation to come in, I think Microsoft embracing Docker in Server 2016 was the shot over the bow that signaled (to me) just how much of a big deal Docker/containers might be. And the more I explore them, the more I see them as my invite to come on in. :slight_smile: What I’ll think of them 3 to 6 months from now? … :slight_smile:

Thanks again for your suggestions!