Dynamic Add Service to Load Balanacer

Archive Rancher 1.x
1

prior to 1.2.0 i was able to add service to a load balancer via the api by calling the addservicelink method via the api, but that doesn’t seem to work any more on the v2-beta api, while it adds it as a link, it’s not adding them to the Balancer Rules.

1 Like
2

The new balancer uses portRules instead of links and…creative label strings. But there is not an idempotent way to add another one.

3

:frowning: any time frame on that, that’s really a feature we need.

4

@vincent

I’ve filed this: https://github.com/rancher/rancher/issues/7065. This has broken our deploy system and I should have checked before upgrading but I didn’t.

5

TBH I think I’d rather push for returning a consistent hash of a resource’s relevant properties that you can send back as an If-Not-Modified header on any PUT. And if the hash doesn’t match when you put return 409.

That would allow safe GET/edit/PUT on any resource rather than adding one-off add/remove actions to individual fields of individual resources.

In the meantime, you may be able to use a selector rule on the balancer and add additional port rules on individual selected services. The latter part isn’t in the UI yet but works through the API/compose.

6

@vincent

Luckily the subdomain name matches the stack name, can you provide me guidance on how to set up such a load balancer?

eg: domain graelsfa4588ent.sugardev.io goes to the stack graelsfa4588ent and container web

7

(Side note, those are probably not the best header and code… Anyway)

Add a balancer with a portRule for each source port that has a selector like balance=me. Then you add the same label to some service(s), and the service has a field for lbConfig where you can add more portRules that will target this service. If this works for your use case it’s actually much nicer than updating the balancer separately because the rules live and die with the service automatically.

http://docs.rancher.com/rancher/v1.2/en/api/v2-beta/api-resources/service/

3 Likes
8

@vincent

OMG that is so awesome, I remove a bunch of code and now it just works. Tell the team they did a great job on the new LB’s!

:smiley:

9

Hi Vincent as i understand the port rule requires to know the external port of the lb.
our use case is that both the load balancer and its connected containers should be randomized on starting the stack and the connection between the load balancer and its connected containers should made in docker compose only via internal ports .
can you please advice on how to implement this in rancher 1.2 it worked in 1.13 via the link functionality.
Thanks!

10

@noamkachko There is not a way to do random published port for a balancer in 1.2.

If your services are http(s)-based then it is likely that using a single balancer with fixed (not-random) published ports and selectors to apply rules to the target services will work for you as I described above.

The connection between the balancer and target containers is (and has been) always over the overlay network and connects to the actual port the process is listening on. You do not need to publish ports on the target service to balance to them. For example if you create a service and set the image to nginx and change nothing else (i.e. no port mapping) you can define a balancer that send traffic to that service on port 80 because nginx listens on port 80, even though the nginx containers are not directly available on any host port.

11

Hi Vincent thanks for your detailed replay.
unfortunately not having the ability to randomize LB port prevents us from upgrading to 1.2 version since it breaks our automation use case. we use pool of machines in our automation and the stack we start contain Load balancer and 2 containers,not having ability to randomize can cause us port collisions if 2 stacks will start on same machine.
can i please file issue on the subject since it seems like behavior change from previous versions that allowed to randomize load balancer ports.
Thanks!

12
1 Like
13

@vincent

As you said “and the service has a field for lbConfig where you can add more portRules”.

Is there any documentation about this? I cannot find where this lbConfig “field” should go
into a compose file. An example would be appreciated.

14

http://docs.rancher.com/rancher/v1.4/en/cattle/adding-load-balancers/#selector

15

Update: Nevermind solved it :slight_smile: Thanks

Can you shed some light on my scenario? (since dynamically updating lb rules is currently not possible or difficult)

I want subdomains like abc.domain.com to target service name “abc” in my given stack. Subdomains will be added dynamically and their respective services are created automatically.

So when a new service is added, load balancer is pre-configured to handle these new subdomains.

16

Hi Vincent, thanks for the Info. I gave it a try. Had some success, but some strange problems:

I wanted to do host based routing and provide the hostname with the service.

I added this to the service in rancher-compose
myservice:
lb_config:
port_rules:
- hostname: "newhost.somedomain.com"
protocol: http
source_port: 80
target_port: 8080

Added a selector matching rule to the loadbalncer, that should route to the service mentioned above.
This gives me the Error “Bad response statusCode [422]. Status [422 status code 422]. Body: [code=MissingRequired, fieldName=serviceId, baseType=error] from [https://…/v2-beta/loadbalancerservices]”

So i try to add the service name, too:
myservice:
lb_config:
port_rules:
- hostname: "newhost.somedomain.com"
protocol: http
source_port: 80
target_port: 8080
service: myservice

This leads to “Cycle detected in path myservice->myservice”

Adding the lb_config to a different service in the same stack and then pointing to myservice from there may work, but seems a little awkward to me.

What am I doing wrong here? Thanks for your time!

17

Hi @tombueng

I’ve been hitting the same issue. It seems the stack and service already have to exist before you apply an lb_config to them.

I’ve gotten around this so far by creating/upgrading the stack, then running the rancher CLI pointing to a special rancher-compose.ingress.yml. That however does not work unless you manually stop the stack during the upgrade - the logs for the service indicate that it is stuck in a loop between ‘Updating service’ and ‘Service already reconciled’.

Sooo, the following steps are the only way I can get this to work:

  1. rancher up --stack my-stack --pull --upgrade -d
  2. rancher up --stack my-stack --rancher-file rancher-compose.ingress.yml --pull --upgrade -d
  3. in a separate shell, as the previous command is still blocking rancher stop --type stack my-stack

Pretty sure this is at least two bugs, unless somebody can confirm otherwise (@vincent?)

18

@tombueng :

I logged an issue on GitHub (https://github.com/rancher/rancher/issues/9372) and @superseb suggested that I remove the source_port from lb_config, which worked. Suggest you try the same?

FYI my config is now completely stripped back to just the aspects that are unique to each service, e.g:

version: '2'
services:
  app:
    scale: 1
    labels: 
      load_balance_http: 'true'
    lb_config:
      port_rules:
      - target_port: 8000
        hostname: helloworld.pensions.sbx.zone

Thanks

Tom

19

Good to know how it works, but lb-config instead of labels can’t be changed on upgrades later. For example to change the hostname.