How to set interval stopping old container when doing in-service upgrade, (for handling zero downtime)

Rancher 1.x
1

Hello, i wonder how to set interval stopping old contianer when doing in-service upgrade? :slight_smile:

my goal purpose is for having zero downtime service upgrade using Rancher.

current implementation:

  1. i use Rancher HAproxy as load balancer + kong (nginx-based) as proxy server + backend microservices (lets say it service A)
  2. i use in-service upgrading, using start_first upgrade strategy

issue that i found:

  1. i found my backend service (backend service is proxied by kong) still got 503 when doing upgrade with this strategy (start new container first, then stop old container)
  2. old container instantly stopped after new container started
  3. i found other service (kong) already could resolving backend service hostname to its new container ip address (i ssume it proves Rancher HAproxy update the network instantly)
  4. but on kong (nginx) error log, its backend service upstream still resolving to old (stopped) container ip address, so it got unreacheable 503
  5. i assume it is because dns-caching in nginx engine
  6. so by reloading kong process, my backend service got 200 again (it clears the nginx dns-cache)

So based on this finding:

  1. i wonder if i could set interval period for pending the old container stopped, after new container started, it would solve my problem (for let the kong dns-caching expired) ??
  2. is there any strategy for my service upgrading?