I looked at Weave, Calico and OpenVswitch as docker plugin.
Weave is an easy to deploy vswitch as a docker container, but with some limitations (plugin looses configuration after restart, but networks registered at docker network, only one subnet can be added). I started testing with weave inside a privileged container and that changed permissions for (user-)docker on RancherOS. No docker command executed without “sudo”. I rebooted to get the normal behavior (user-docker as rancher user) back.
OpenVswitch is a powerful but complex solution I haven’t used so far.
Calico is a vRouter solution. With my test setup create a profile (network) and append containers to it looks good, but I can’t ping containers inside the same profile / network. The icmp packet reached the host, but it isn’t routed to the destination container. Containers get a /32 ip address and you can’t use the same ip twice (because it would be a routing conflict, calico will deny duplicates). Really quick test on CentOS7 with Docker 1.9 (dev) yesterday.
I like calico, but you can’t build independent container networks for different customers with the same subnet.
Is there a documentation how rancher network works and how it could be used with custom network needs (manual assigned ips, different service / stack subnets, …)?
I found this discussion about custom network configuration here in the forum.
http://forums.rancher.com/t/how-to-launch-with-custom-network-configuration/83