Internal Load Balancer weird behaviour

Rancher 1.x
1

Hello,

I have a question regarding the behaviour of the load balancer

I have link service like 1 LB load balanced on 5 containers

BUT when I edit the only 1 LB I have it’s configuration says that it has only one backend

and obviously this only container get the traffic but not the 4 others…

What am I missing ??

thanks

sebastien

2

I have more than 1848 entries doing the request

select * from config_item_status where name=‘haproxy’;

| 13407 | haproxy |                 7 |               7 | 3e0d61c5cd2851e80416ea7bdd7aa59b417a3f11fa7852d0f0f91a26a6cdadb9 | 2016-02-01 19:45:44 | 2016-02-01 19:45:45 |     1847 |       NULL |       NULL |        1847 | agent_id      |
| 13408 | haproxy |                 7 |               7 | 3e0d61c5cd2851e80416ea7bdd7aa59b417a3f11fa7852d0f0f91a26a6cdadb9 | 2016-02-01 19:45:44 | 2016-02-01 19:45:45 |     1848 |       NULL |       NULL |        1848 | agent_id      |

this request also give me 1555 rows
select * from instance where agent_id in (select agent_id from config_item_status where name=‘haproxy’);

it semas to store a lot of very out of date datas in there ? is it possible that slow down my internal network ?? (take sometimes more than 4 seconde to reach another container)

Is there a way to purge the Rancher Server DB content (execpt the global configuration / users, credentials, environments… ??

3

You should be editing load balancers at the service level and not at the container level.

You can check the backend of the load balancer by exec-ing into the LB container and displaying the /etc/haproxy/haproxy.cfg and you’ll be able to see all the containers that are going to show traffic. It will display all the IPs of all the containers of the service that you are load balancing.

4

thanks @denise inside the loadbalancer configuration seams to be good, but the LB stay stuck in Initializing state:

Pasted image523×527 20.1 KB

and know it always respond 503…

I’ve ssh into the LB and installed telnet and it seams that the backend is not reachable

backend 2cde9c4a-feda-4ba8-85c7-d486fee6b863_80_1_backend
mode http
server 05bb63ed-a6fc-45c9-b7fd-698e9350e478 10.42.237.94:80
server bec9787b-3703-4340-98b0-49a568dedfe4 10.42.70.17:80
server 2396c507-33a4-4f97-9bf0-404c72fcbaff 10.42.239.153:80
server db4701d6-a37d-40ad-a30d-2562f5a23653 10.42.151.66:80
server fef5004d-818a-4cee-8e6b-8f4dbe363f59 10.42.111.30:80

root@454332e81b9e:/# telnet 10.42.237.94 80
Trying 10.42.237.94…

I still have some others backend that are responding

 root@454332e81b9e:/# telnet 10.42.203.5 80
 Trying 10.42.203.5... 
 Connected to 10.42.203.5. 
 Escape character is '^]'.

I don’t know how I can debug that, it was working fine yesterday, and today network seams unreachable…
I’ve tester the request directly on the backend and it is working.
Is there any change in the overlay network ?

I’m using v0.56.1

Thanks
sébastien

5

on my stack, all my LB containeurs are in bad state while my services seams good and responsive

6

Hi,

I’ve fixed all my weird behaviour which where relative to ipsec tunneling by completely remove my rancher-server (delete /var/lib/mysql directory) and make up a new fresh install of v0.56.1.

all works perfectly then!

sebastien