Using Apache with public IP in front of nginx-ingress in private rancher cloud

Rancher
1

Hi, we have a working 2.2.x rancher setup on local hardware, serving several web applications.
The cluster has the standard nginx-ingress configured via
Edit cluster -> Advanced Options -> Nginx Ingress -> enabled.

Just to recap how I understood things work: there is one default-http-backend
running the container rancher/nginx-ingress-controller-defaultbackend
on (any) one of the worker nodes. Initially this could be 10.0.X.Y
Since the K8S cluster does not have a public IP, we use an Apache httpd
with RewriteRules to pass all traffic for mywebapp to 10.0.X.Y.

The mywebapp.yaml now has a host rule: mywebapp.10.0.X.Y.nip.io
forwarding to backend: serviceName: mywebapp-service.

My issue is that if default-http-backend dies, it can be re-started on
any other worker node like 10.0.X.Z, and both the apache RewriteRule
and the mywebapp.10.0.X.Y.nip.io would use the wrong IP address.

=> Am I using the wrong approach here ? Could someone point me
to best-practices for this scenario ?

Thanks in advance, yours, Steffem

1 Like
2

@sneumann, Ingress Pods by default are a Daemon set and there will be one per node. Using any ingress that forward traffic to a specific IP violates the nature of what Kubernetes is supposed to provide. The Ingress controller pods take the http(s) route and forward to a specific backend Kubernetes Service. Your rewrite rules will need to be dynamic. Or you can turn off our nginx ingress and implement your own using Pod Affinity and Anti-Affinity rules to force the various pods where you want them. https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

3

Hi, thanks for the quick reply.

Yes, I can see the nginx-ingress-controller, one for each node. I had thought that I need to pass external traffic to default-http-backend which would then route to the actual service, and indeed this would be a specific IP and not the K8S way to go, hence my posting here.

Let me ask the question differently: I have one public IP with an Apache httpd, and the nginx-ingress inside the rancher. What kind of configuration do I need in the Apache to pass traffic into the nginx-ingress ?

Currently I use an Apache
RewriteRule ^/mywebapp/(.*) http://mywebapp.10.0.X.Y.nip.io/mywebapp/$1 [P,L]
and I want to get rid of the fixed 10.0.X.Y. Do I need to configure the Apache as load-balancer with mod_proxy, pointing to all IPs where the default-http-backend could be deployed ? Most K8S documentation I found assumes I use a cloud provider where I can get my external load balancer.

Yours, Steffen

kubectl get pods -n ingress-nginx
NAME                                    READY   STATUS    RESTARTS   AGE
default-http-backend-5954bd5d8c-w56r5   1/1     Running   0          6h54m
nginx-ingress-controller-5djr5          1/1     Running   0          6h50m
...
nginx-ingress-controller-9bjwl          1/1     Running   0          6h50m
nginx-ingress-controller-9bzqf          1/1     Running   0          6h53m
4

The default backend is only to handle unmatched routes. All traffic should be load balanced nodes with the nginx-ingress-controller-* pods.