HAProxy -> Apache X-Forwarded-For not working

Rancher 1.x
1

So I’m completely at a loss. I’ve literally spent the last 40 hour work week trying to fix this issue and I’m close to giving up. As a Hail Mary I’m hoping someone smarter than me can help me see my mistakes.

The issue: I have Apache 2.4 behind a Rancher HAProxy 1.6 Load balancer and can’t get Apache to recognize the real client ip in the logs; always showing the ip of HAproxy.

Here’s my current HAProxy.cfg

defaults
    maxconn 4096
    mode tcp
    option forwardfor
    option http-server-close
    option redispatch
    retries 3
    timeout client 50000
    timeout connect 5000
    timeout server 50000

frontend 80
  bind *:80
  mode http
  default_backend 80

backend 80
  acl forwarded_proto hdr_cnt(X-Forwarded-Proto) eq 0
  acl forwarded_port hdr_cnt(X-Forwarded-Port) eq 0
  option forwardfor
  http-request add-header X-Forwarded-For %[src]
  http-request add-header X-Client-IP %[src]
  http-request set-header X-haproxy-custom "castle9mm"
  http-request add-header X-Forwarded-Port %[dst_port] if forwarded_port
  http-request add-header X-Forwarded-Proto https if { ssl_fc } forwarded_proto
mode http

Here’s my apache virtualhost
RemoteIPHeader X-Forwarded-For RemoteIPInternalProxy 10.42.40.172 RemoteIPTrustedProxy 10.42.40.172 

<IfModule mod_headers.c>
  Header set X-apache-custom "castle9mm"
</IfModule>

ErrorLog ${APACHE_LOG_DIR}/error.log
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog ${APACHE_LOG_DIR}/access.log combined env=!dontlog

I’ve tried a number of different log formats and I don’t know which is correct. I’ve read that mod_remoteip should automatically replace the %h variable with the correct client ip, I’ve also read you need to replace the %h variable with %a, I’ve also read you need to replace %h with %{X-Forwarded-For}i. Nothing i tried worked.

When using chrome inspector, the only header I see is the “X-apache-custom”. I’m not even seeing the “X-haproxy-custom” header so I’m not sure if haproxy is even working right. I’ve also tried replacing the %[src] variable in haproxy.cfg with a hard-coded IP just to see if it gets set but that didn’t work either.

full disclosure: I have stripped out identifying container info in the configs. I don’t think that should matter but there may be a typo.

Any help or insight would be greatly appreciated as I feel like I’m taking crazy pills.

1 Like
2

Have you found something ?