Rancher metadata slow to respond on container start

Archive Rancher 1.x
1

We are building containers using spring boot and eureka. Inside Rancher we can’t resolve the hostname of the container (the eureka default) so we register using the rancher ip. The following script does this and is designated the ENTRYPOINT. We then build our application image and pass CMD as java -jar myapp.jar. This should append --eureka.instance.hostname= to the command.

Sadly the curl doesn’t work the first time, and only the first time so it doesn’t look like a timing issue. Can anyone explain why?

#!/bin/bash

# this script is designed as a wrapper for a spring boot application which may or may
# not run under Rancher. In Rancher we need to explicitly set the Eureka instance name
# to our Rancher IP which resolves within the Rancher world. On the outside we don't.

# BODGE - rancher-metadata doesn't work the first time.
curl -s http://rancher-metadata/latest/self/container/primary_ip > /dev/null

EUREKA_INSTANCE_HOSTNAME=$(curl -s http://rancher-metadata/latest/self/container/primary_ip)
if [ $? -eq 0 ]; then
    OPTS="--eureka.instance.hostname=$EUREKA_INSTANCE_HOSTNAME"
fi

# now hand over to the CMD[] string. Use exec so that the running process is pid 1 and
# will get signals correctly from docker
exec $* $OPTS
2

kirboro, this happens because Rancher hasn’t finished setting up the network by the time docker calls the entrypoint. It usually takes a little bit to come up. Depending how patient you are, you could throw in a sleep 30 at the beginning, or wrap the curl in a while loop with a sleep 1 or something.

Once you get a response, you can assume you have Rancher metadata access and DNS.

3

Thought that might be the case but wanted to check. Adding a short wait is definitely the wrong answer as there will be one day it fails again. I guess I wrap this in a loop.

4

I believe we can fix this with Docker 1.9 now.

But checking and waiting for dependencies is a good idea in general. e.g. if you link service A to B, we may start up B before A but there’s no guarantees that the daemon you need on B is up and responding before A tries to hit it.

5

Thanks for that. In this case we just need to know who we are to advertise the right way to Eureka. The actual service will then connect itself asynchronously and will reconnect if things change.