Hi I am using RancherOS newly, and most things work fine but the System docker creates a vir Interface named docker-sys that is in a private address range (some 172.18… /16 net) but that causes major routing problems with my existing network structure as I have alot of 172.18. sub nets in it. and Rancher needs to pull some informations from machines in that net and cant get a route to it, I tried configuring the System-docker.json like I did with the daemon.json for the user docker but RancherOS seems to ignore these settings when launiching the system-docker.
I also tried to manage some route manually but that is very troublesome and not rly troublefree
Anyone have a fix?
Hi,
See the three tickets below.
docker-sys purpose is for the system-docker in rancher, different interface than user docker (this is mentioned in 2478, the best ticket for this).
The 2478 gives latest info:
opened 09:21PM - 12 Sep 18 UTC
closed 06:01AM - 29 Sep 18 UTC
kind/question
**RancherOS Version: (ros os version)**
```sh
[rancher@server ~]$ ros -v
ve… rsion v1.4.0 from os image rancher/os:v1.4.0
```
**Where are you running RancherOS? (docker-machine, AWS, GCE, baremetal, etc.)**
OpenStack
This is potentially a bug and I'm opening this issue for clarification of expected behavior with the new way to configure the system-docker bridge network. I've already reviewed https://github.com/rancher/os/issues/2329 and https://github.com/rancher/os/issues/2422.
After following the instructions in the above issues, I've opened a new one because I believe the network isn't being properly configured. At least it's not being configured how I would expect it.
My cloud-config:
```yaml
#cloud-config
rancher:
docker:
extra_args:
- "--bip=192.169.0.1/16"
system_docker:
extra_args:
- "--bip=192.170.0.1/16"
```
The above cloud-config produces the user-docker network as I expect. Output below.
```
[rancher@server ~]$ ifconfig docker0
docker0 Link encap:Ethernet HWaddr 02:42:85:BE:5D:2E
inet addr:192.169.0.1 Bcast:192.169.255.255 Mask:255.255.0.0
...
```
But it doesn't configure the system-docker network as I expect. Output below:
```
[rancher@server ~]$ ifconfig docker-sys
docker-sys Link encap:Ethernet HWaddr 02:42:6F:2D:38:1C
inet addr:192.170.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
...
```
I would expect that the broadcast address (`Bcast`) for the `docker-sys` interface would be set just like it is for the `docker0` interface.
1. Is this expected behavior?
2. If not, are their workarounds?
* The only one we've come up with is running `ifconfig` on the server to properly configure the broadcast address for the interface.
opened 07:43AM - 16 Apr 18 UTC
closed 01:39PM - 17 May 18 UTC
kind/enhancement
area/ros
area/documentation
status/to-test
Before v1.3, we used CNI to configure docker-sys.
In v1.3, we removed docker-… sys and let system-docker use only the host network, but some users want to be able to keep the bridge network for system-docker.
So we need to re-introduce docker-sys, but we need to remove the CNI style and use the docker args style.
opened 07:52AM - 26 Jul 18 UTC
closed 09:24AM - 26 Jul 18 UTC
**RancherOS Version: (ros os version)** v1.4.0
**Where are you running Rancher… OS? (docker-machine, AWS, GCE, baremetal, etc.)** VMware esxi 6
Hi,
When I upgraded from v1.3.0 to v1.4.0, docker-sys & br-* interface's network addresses are in conflict.
```
rancher@ros1*:~:-) ip r
default via 192.168.101.254 dev eth0
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev docker-sys proto kernel scope link src 172.18.42.1 linkdown
172.18.0.0/16 dev br-d214f2215687 proto kernel scope link src 172.18.0.1 linkdown
172.21.0.0/16 dev eth2 proto kernel scope link src 172.21.0.33
192.168.16.0/24 dev eth1 proto kernel scope link src 192.168.16.91
192.168.101.0/24 dev eth0 proto kernel scope link src 192.168.101.91
```
I searched workaround and found https://github.com/rancher/os/issues/1870 .
So, I reverted to the snapshot before upgrade and tried upgrade again with the command below.
```
sudo ros os upgrade --append 'rancher.system_docker.subnet=172.22.42.1/16'
```
But network address didn't change after reboot.
I checked config with 'sudo ros config export' and found the correct config below.
```
rancher:
system_docker:
subnet: 172.22.42.1/16
```
But, bridge config is not changed.
```
rancher@ros1*:~:-) cat /etc/docker/cni/bridge.d/bridge.conf
{
"name": "bridge",
"type": "bridge",
"bridge": "docker-sys",
"isDefaultGateway": true,
"ipMasq": true,
"hairpinMode": true,
"ipam": {
"type": "host-local",
"subnet": "172.18.42.1/16"
}
}
```
Next , I followed workaround https://github.com/rancher/os/issues/1870#issuecomment-356492220 .
I edit /etc/docker/cni/bridge.d/bridge.conf & reboot -f.
```
- "subnet": "172.18.42.1/16"
+ "subnet": "172.22.42.1/16"
```
but network config is still not changed. (the same as first one).
```
rancher@ros1*:~:-) ip r
default via 192.168.101.254 dev eth0
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev docker-sys proto kernel scope link src 172.18.42.1 linkdown
172.18.0.0/16 dev br-d214f2215687 proto kernel scope link src 172.18.0.1 linkdown
172.21.0.0/16 dev eth2 proto kernel scope link src 172.21.0.33
192.168.16.0/24 dev eth1 proto kernel scope link src 192.168.16.91
192.168.101.0/24 dev eth0 proto kernel scope link src 192.168.101.91
```
How can I solve address conflict ?