Hello, i wonder how to set interval stopping old contianer when doing in-service upgrade?
my goal purpose is for having zero downtime service upgrade using Rancher.
current implementation:
- i use Rancher HAproxy as load balancer + kong (nginx-based) as proxy server + backend microservices (lets say it service A)
- i use in-service upgrading, using
start_first
upgrade strategy
issue that i found:
- i found my backend service (backend service is proxied by kong) still got 503 when doing upgrade with this strategy (start new container first, then stop old container)
- old container instantly stopped after new container started
- i found other service (kong) already could resolving backend service hostname to its new container ip address (i ssume it proves Rancher HAproxy update the network instantly)
- but on kong (nginx) error log, its backend service upstream still resolving to old (stopped) container ip address, so it got unreacheable 503
- i assume it is because dns-caching in nginx engine
- so by reloading kong process, my backend service got 200 again (it clears the nginx dns-cache)
So based on this finding:
- i wonder if i could set interval period for pending the old container stopped, after new container started, it would solve my problem (for let the kong dns-caching expired) ??
- is there any strategy for my service upgrading?