Whitelist in ingress not working

I am running Rancher 2.5.7 ontop of k3s on Fedora CoreOS.

I have tried a couple different versions (Found several different things via google):
traefik.ingress.kubernetes.io/whitelist-source-range
ingress.kubernetes.io/whitelist-source-range
nginx.ingress.kubernetes.io/whitelist-source-range

But none of these seem to work. I have found if I whitelist the internal container network (10.42.1.x), the whitelist works. But if I whitelist my home internal network (172.20.30.x), I get forbiddens. So it seems to only see the kubernetes internal network, even when coming from a computer on my internet home LAN.

I think I figured this out…

I think I needed to patch my traefik service:

kubectl patch svc traefik -p '{"spec":{"externalTrafficPolicy":"Local"}}' -n kube-system