I opened an issue on GitHub for this but maybe it’s only a configuration issue on my side:
Basically, i can’t make service links work when services are located on different hosts. For example:
Zookeeper running on host A,
Mesos-Master running on host B, with service link to Zookeeper service.
Host A:
VirtualBox machine create with docker-machine running boot2docker, agent added via custom host registration, 192.168.99.100 as IP being registered.
Host B:
VirtualBox machine create with docker-machine running boot2docker, agent added via custom host registration, 192.168.99.101 as IP being registered.
Both VMs are running on my local machine (192.168.0.xx). Rancher server is running on my local machine. It detects both agents and registers them fine. Containers are launched successfully, but somehow they are unable to communicate with each other when running on different hosts.
I could finally manage to get the cross-host network working by trying out with 2 bare-metal machines hosts on the same local area network. It just worked fine.
Setting up hosts on virtualbox is not possible due to NAT network configuration. Even in bridged mode, i couldn’t manage to make them work properly. I managed to run a hybrid configuration with one local host and one AWS EC2 host and it worked fine.
@jloisel, haven’t read your posts, just wanted to say that I’ve used virtualbox without issues, using docker-machine to provision the hosts, then standard docker run's to start rancher server and agent. So, it’s most likely something with your setup.
I provisioned two hosts with docker-machine too but i couldn’t manage to get containers communicate between those hosts. By default, it configures both boxes with NAT. I can’t setup NAT for 500/udp and 4500/udp for both machines since it would conflict on my local machine network.
I’ve tried with bridged network and it didn’t work either. I don’t know why, everything seemed fine when inspecting IPTables. Maybe one of my network interfaces on my local machine is messing up the IPSec tunnel.
have you tried making two containers communicate with 2 virtual boxes with NAT networking? I tried to setup NAT port forwarding on both VMs and it didn’t work.
I`ve the same issues when trying to run more that one VM on the same host. I was trying to get them up using docker-machine and then adding them manually as rancher hosts. At some point linking stops working.
Maybe the Rancher team could clarify the network infrastructure and if that use case is really not supported.
Ah, right, no I’ve only used two nodes in virtualbox, one for the server and the other a lone agent node. With a dual agent setup, I get the same results as the rest of you.
It’s simply not possible to setup two nodes on virtualbox vms on the same machine unless you configure them with bridged networking. NAT cannot be used since it would require to forward the UDP ports to the local machine. You cannot forward those ports for both the VMs on the same machine.
But, somehow bridged network is not working properly with boot2docker VMs. It freezes my entire Ubuntu setup for an unknown reason.
