Adding HA / VIP Ingress for non-VM deployments

I have Harvester 1.3.0 on 3 node cluster. Each node has 2 Ethernet ports.
Installed in the office LAN network

The Harvester API and UI is configured to static IP

And I’m running deployments (few web servers) right on default Kubernetes (Same cluster where Harvester is running VMs).

At first I used default (included) Ingress, but wanted the apps traffic to flow through the non-management network.

So, here what I did:

  • created new cluster network (and VM net just in case)
kind: NetworkAttachmentDefinition
  annotations: '{"mode":"auto","serverIPAddr":"","cidr":"","gateway":""}'
  labels: cn2 'true' UntaggedNetwork
  name: net2
  namespace: default
  config: >-
  • Installed Traefik via Helm Chart:
helm repo add traefik
helm repo update
helm install traefik traefik/traefik -f

As you can see above, I have my values defined here:

The important part is:

  enabled: true
  single: true
  type: LoadBalancer

I configured Traefik to create LoadBalancer, which relies on kube-vip, included already with Harvester.

Everything deployed successfully. Tested with few deployments.

But it looks like the VIP, which kube-vip created for Traefik still uses same management network:

$ arp -a
? ( at 02:11:32:2a:16:9a [ether] on enp1s0
? ( at 02:11:32:2a:16:9a [ether] on enp1s0

So, how can I configure LoadBalancer to use non-management network?

Finally figured how to add additional network interface from non-management network to the pod.

For Traefik, here is how I do that:

    # '[ { "name": "net2", "namespace":"default" } ]'

But… then I realized I don’t need that non-management NIC on the Traefik pod. I need it on VIP LoadBalancer!

So, looked at the kube-vip config. And the one deployed with Harvester by default doesn’t include additional NIC. Which probably makes sense, because kube-vip is part of the system installation, and there is no additional network exists during installation - they added later.

So, does that mean that I need to modify system-installed kube-vip to add non-management NIC to it? Or would be more appropriate to install second instance of kube-vip to work on non-management NIC?

Is there any other solution to enable HA Ingress on non-management NIC?