Network MTU, Bonding, Bridge and VLAN example

RancherOS
1

Hi All,

I’m investigating Rancher and have made some progress in understanding the network config, however the docs still seem to be a work in progress. I’ve been able to piece this together:

    hostname: hostname
rancher:
  console: fedora
  docker:
    engine: docker-17.05.0-ce
    tls: false
  network:
    interfaces:
      bond0:
        bridge: br0.10
        bridge: br0.40
        vlans: 10,40
        bond_opts:
          downdelay: "100"
          lacp_rate: "1"
          miimon: "100"
          mode: "4"
          updelay: "100"
          xmit_hash_policy: layer3+4
          mtu: 9000
      mac=00:11:55:d2:46:80:
        bond: bond0
      mac=00:11:55:d2:46:81:
        bond: bond0
      mac=00:11:55:d2:46:83:
        bond: bond0

      br0.10:
        bridge: true
        dhcp: false
      br0.40:
        bridge: true
        address: 10.40.0.21/16
        pre_up:
          - ip addr del 10.40.0.21/16 dev br0.40
          - ip addr add 10.40.0.21/16 dev br0.40 broadcast +

      eth7:
        address: 10.1.0.21/24
        gateway: 10.1.0.1
        mtu: 9000
        dhcp: false
    dns:
      nameservers:
      - 10.1.0.10
      - 10.1.0.11
ssh_authorized_keys:
  - ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlz...w== erric@hostname

The interfaces are created, I get a bond, a vlan and a bridge interfaces:

bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
        inet6 fe80::226:55ff:fed2:4681  prefixlen 64  scopeid 0x20<link>
        ether 00:26:55:d2:46:81  txqueuelen 1000  (Ethernet)
        RX packets 25468  bytes 2840564 (2.7 MiB)
        RX errors 0  dropped 414  overruns 0  frame 0
        TX packets 509  bytes 60902 (59.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@virt1 ~]# ifconfig bond0.10
bond0.10: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::226:55ff:fed2:4681  prefixlen 64  scopeid 0x20<link>
        ether 00:26:55:d2:46:81  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15  bytes 1138 (1.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@virt1 ~]# ifconfig bond0.40
bond0.40: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::226:55ff:fed2:4681  prefixlen 64  scopeid 0x20<link>
        ether 00:26:55:d2:46:81  txqueuelen 0  (Ethernet)
        RX packets 2075  bytes 103810 (101.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16  bytes 1180 (1.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@virt1 ~]# ifconfig br0.10  
br0.10: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::d4:99ff:fe69:2dc  prefixlen 64  scopeid 0x20<link>
        ether 02:d4:99:69:02:dc  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15  bytes 1138 (1.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@virt1 ~]# ifconfig br0.40
br0.40: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.40.0.21  netmask 255.255.0.0  broadcast 0.0.0.0
        inet6 fe80::f04b:e2ff:fe7b:2f29  prefixlen 64  scopeid 0x20<link>
        ether f2:4b:e2:7b:2f:29  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 19  bytes 1334 (1.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I’m not sure why the MTU isn’t getting set. I’ve tried specifically adding individual interfaces with an MTU setting, for example:

eth4:
  mtu: 9000

I have also tried:

mac=de:ad:be:ef:00:01
  mtu: 9000

However this doesn’t work either. Any tips/cluebats? Is this possible to do without a pre_up statement? Any tips for reordering this file around?

I was also concerned about the broadcast address not showing properly, however I did see this thread which should take care of things in the next release.

2

I was trying to set the mtu in the v1.1.0-rc3 gce image yesterday, and I have this sinking feeling that I need to make some mildly serious changes to make it work correctly.

Once v1.1.0 is out, I’ll be looking into it - unless someone beats me to it.

3

Thanks @SvenDowideit I don’t do too much programming, so i doubt I’ll be able to dig into this in a meaningful way. I do like Rancher vs Core0S, it seems to make more sense, have a better thought out install / interface.

I am happy to try out any patches or short cuts you may have.

E