Question about Private Networks

Rancher 1.x
1

Hi everyone

I am trying to setup Rancher-Server & Hosts on Exoscale. I have a certain configuration in mind and am unsure if it’s wise and/or possible. I guess the same would apply to DigitalOcean (they also offer private networks).

First my tl;dr final goal: Have rancher-server and all hosts communicate between each other over a private network but allow services to expose some ports to WAN. I would also like 1 rancher-agent on the same server as my rancher-server to host environment wide services (external dns & let’s encrypt).

In more details this leads me to the following questions:

  • On Exoscale I defined a private network and provisioned my rancher-server host to have eth1 with an IP on this network. I would like for rancher/docker to communicate over this private network if possible. So it will setup it’s ipsec network over that and not the public network. Is this possible? How could I achieve this? (also in combination with question 2)
  • My rancher-server Host only has Ports 22 (restricted to office ip),80,443 exposed (WAN) and my porivisioned host has the default rancher-machine security gorup (22, 2376, 500, 4500, ICMP ping) open (WAN). It “seems” to work like this but is this advisable like this? If I launch a service on the provisioned host which accepts traffic from WAN so I configure a loadbalancer in front of it to accept port e.g. 443 I would also need to expose port 443 on my rancher-machnine group, right?
  • I would like to start a rancher-agent on the same host as my rancher-server is running. There it asks that I expose ports 500, 4500 for it, is that necessary if it’s on the same host (I also have other hosts provisioned) or is this necessary to communicate with the other hosts? I also asks for a public IP, especially in relation to the first question should this be the public or private netwrok ip? Would this interfere with the firewalling rules in question 2?

I also have some issues with the Exoscale driver:

  • When I select “CoreOS” to be porvisioned it always provisions an Ubuntu server.
  • I can’t find an option to provision with a cloud-config file so I have no clue how to even prepare the network interface.

Here is my example cloud-config I would use for the hosts and the server:

#cloud-config
hostname: HOSTNAME

users:
  - name: USERNAME
    ssh-authorized-keys:
      - ssh-rsa ...
    groups:
      - sudo
      - docker
  
coreos:
  update:
    reboot-strategy: off
    group: stable
  units:
    - name: settimezone.service
      command: start
      content: |
        [Unit]
        Description=Set the time zone
  
        [Service]
        ExecStart=/usr/bin/timedatectl set-timezone Europe/Zurich
        RemainAfterExit=yes
        Type=oneshot
    - name: 00-eth1.network
      runtime: true
      content: |
        [Match]
        Name=eth1
  
        [Network]
        Address=192.168.1.X/24
  
write_files:
  - path: /etc/systemd/timesyncd.conf
    content: |
      [Time]
      NTP=0.ch.pool.ntp.org 1.ch.pool.ntp.org
  - path: /etc/ssh/sshd_config
    permissions: 0600
    owner: root:root
    content: |
      # Use most defaults for sshd configuration.
      UsePrivilegeSeparation sandbox
      Subsystem sftp internal-sftp
  
      PermitRootLogin no
      AllowUsers core einsrnch01
      PasswordAuthentication no
      ChallengeResponseAuthentication no