I have Hyper-V VM with two network interfaces, that have to be configured manually (no DHCP). I can boot RancherOS from ISO in VM and configure networking from console with ifconfig/ip rule/ip route and it works. But when I put required settings to cloud-config.yml and install RancherOS to disk usung this file, setings are not applied.
After installation, ros config get rancher.network shows all settings, but in ifconfig/ip rule/ip route output there are only defaults - nothing is applied from cloud-config.yml and network is not available.
I’ve tried RancherOS 0.5.0 and 0.4.5, but not luck - I can’t get it working. Is there something wrong with my cloud config file?
#cloud-config
ssh_authorized_keys:
- ssh-rsa AAAA...==
hostname: HVRancher01
write_files:
- path: /etc/iproute2/rt_tables
permissions: 0644
owner: root
content: |
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
15 Vlan15
201 Vlan201
rancher:
network:
dns:
override: true
nameservers:
- 11.201.15.1
- 8.8.8.8
- 8.8.4.4
search:
- xxx.company.com
interfaces:
eth*:
dhcp: false
eth0:
match: eth0
address: 11.201.15.20
gateway: 11.201.15.254
post_up:
- ip route add 11.201.15.0/24 dev eth0 table Vlan15
- ip route add default table Vlan15 via 11.201.15.254
- ip route add default via 11.201.15.254 dev eth0
eth1:
match: eth1
address: 11.201.201.20
post_up:
- ip route add 11.201.11.0/24 via 11.201.201.254
- ip route add 11.201.254.0/24 via 11.201.201.254
- ip route add default table Vlan201 via 11.201.201.254
pre_cmds:
- ip rule add from 11.201.201.0/24 table Vlan201
- ip rule add from 11.201.15.0/24 table Vlan15
services:
rancher-server:
image: rancher/server
restart: always
ports:
- 8080:8080
Ok, this is ugly as hell, but I’ve been able to provision RancherOS using start.sh script:
#cloud-config
ssh_authorized_keys:
- ssh-rsa AAAA...==
hostname: HVRancher01
write_files:
- path: /etc/iproute2/rt_tables
permissions: "0644"
owner: root
content: |
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
15 Vlan15
201 Vlan201
- path: /opt/rancher/bin/start.sh
permissions: "0755"
owner: root
content: |
#!/bin/bash
#
# routing rules
ip rule add from 11.201.201.0/24 table Vlan201
ip rule add from 11.201.15.0/24 table Vlan15
#
# interface configuration
ifconfig eth1 11.201.201.20 netmask 255.255.255.0
ifconfig eth0 11.201.15.20 netmask 255.255.255.0
#
# routes
ip route add 11.201.15.0/24 dev eth0 table Vlan15
ip route add default table Vlan15 via 11.201.15.254
ip route add default via 11.201.15.254 dev eth0
ip route add 11.201.11.0/24 via 11.201.201.254
ip route add 11.201.254.0/24 via 11.201.201.254
ip route add default table Vlan201 via 11.201.201.254
#
# end
rancher:
network:
dns:
override: true
nameservers:
- 11.201.15.1
- 8.8.8.8
- 8.8.4.4
search:
- xxx.company.com
services:
rancher-server:
image: rancher/server
restart: always
ports:
- 8080:8080
I still hope that there is a better way to do this…
Finally, I’ve figured that out: IP addresses have to be in CIDR notation, eg: address: 11.201.15.20/24
interfaces:
eth*:
dhcp: false
eth0:
match: eth0
address: 11.201.15.20/24
gateway: 11.201.15.254
post_up:
- ip route add 11.201.15.0/24 dev eth0 table Vlan15
- ip route add default table Vlan15 via 11.201.15.254
- ip route add default via 11.201.15.254 dev eth0
eth1:
match: eth1
address: 11.201.201.20/24
post_up:
- ip route add 11.201.11.0/24 via 11.201.201.254
- ip route add 11.201.254.0/24 via 11.201.201.254
- ip route add default table Vlan201 via 11.201.201.254
I’ll just keep updating this, maybe it’ll help someone. Another gotcha: /etc/iproute2/rt_tables has to be in the network containter, created by system-docker. It means that I can’t use write_files, because it creates files in the console container.
Here is the workaround:
#cloud-config
ssh_authorized_keys:
- ssh-rsa AAAA...==
hostname: HVRancher01
rancher:
network:
dns:
override: true
nameservers:
- 11.201.15.1
- 8.8.8.8
- 8.8.4.4
search:
- xxx.company.com
interfaces:
eth*:
dhcp: false
eth0:
match: eth0
address: 11.201.15.20/24
gateway: 11.201.15.254
post_up:
- ip route add 11.201.15.0/24 dev eth0 table Vlan15
- ip route add default table Vlan15 via 11.201.15.254
- ip route add default via 11.201.15.254 dev eth0
eth1:
match: eth1
address: 11.201.201.20/24
post_up:
- ip route add 11.201.11.0/24 via 11.201.201.254
- ip route add 11.201.254.0/24 via 11.201.201.254
- ip route add default table Vlan201 via 11.201.201.254
pre_cmds:
- bash -c "echo -e '#\\n# reserved values\\n#\\n255 local\\n254 main\\n253 default\\n0 unspec\\n#\\n# local\\n#\\n#1 inr.ruhep\\n15 Vlan15\\n201 Vlan201' > \/etc\/iproute2\/rt_tables"
- ip rule del from 11.201.201.0/24 table Vlan201
- ip rule del from 11.201.15.0/24 table Vlan15
post_cmds:
- ip rule add from 10.200.13.0/24 table Vlan15
- ip rule add from 10.200.101.0/24 table Vlan201
services:
rancher-server:
image: rancher/server
restart: always
ports:
- 8080:8080