Multiple Cluster connectivity issue on vSphere vms


I am trying to deploy multiple k3s clusters on VM’s running in vSphere (3 esxi hosts).

My issue is that only one cluster is actually able to communicate with the outside of the cluster.

On the failing clusters coredns is failing, at least I am getting an timeout when trying to lookup anything.
On the cluster that works, I have gotten a flannel and cni0 interfaces on all nodes.
On the failing clusters one master node has both flannel and cni0, the rest has a mixture of no cni0 or no flannel AND cni0

I have tried setting flannel up with both vxlan, host-gw and ipsec, with no luck.

I am suspecting that there is something in vSphere that is misconfigured/not configured. The question is just what could that be?

The clusters are deployed using Fully Automated K3S etcd High Availability Install | Techno Tim Documentation

Any ideas? or are there missing some information?