Debugging service issues with Rancher-Compose

Archive Rancher 1.x
1

Currently running Rancher 0.47 + compose v0.5.3 and just tried to deploy my first service from a private repository… I suspect I’m currently facing an issue related to authentication with the private repository, but I don’t really know how to debug the issue to verify.

I currently see the error: State: Removed ( : c8dbb906-0374-4f25-8186-c92d6f362c3d : Operation failed) in the UI and the container is shown as jumping between activating and this state. I’m not entirely sure how to go beyond this in terms of debugging. My question is, what can I do to debug this further?

Here’s what I’ve done so far:

When I jump on the host box, I don’t see any stopped containers or the docker image present on the host. When I run rancher-compose logs it just outputs

ERRO[0001] Failed to get logs for my-first-docker_web_1: Failed to find action: logs

When I’m actually running rancher-compose up -d I see something like this:

DEBU[0000] Environment Context from file : map[]
DEBU[0000] Opening compose file: docker-compose.yml
DEBU[0000] [0/0] [web]: Adding
DEBU[0000] Opening rancher-compose file: /Users/untoldone/Source/my-first-docker/rancher-compose.yml
DEBU[0000] Looking for stack my-first-docker
INFO[0000] Creating stack my-first-docker
DEBU[0001] Launching action for web
DEBU[0001] Project [my-first-docker]: Creating project
DEBU[0001] Finding service web
DEBU[0001] [0/1] [web]: Creating
INFO[0001] Creating service web
DEBU[0001] Creating service web with hash: digest.ServiceHash{Service:"ef6f20e9c2931390034d1561104d5547416b525f", LaunchConfig:"1769b015c601991cff358ef2697a68ac53ad5918", SecondaryLaunchConfigs:map[string]string(nil)}
DEBU[0001] Finding service web
DEBU[0001] Found service web
DEBU[0001] [0/1] [web]: Created
DEBU[0001] Project [my-first-docker]: Project created
INFO[0001] Project [my-first-docker]: Starting project
DEBU[0001] Launching action for web
DEBU[0001] Finding service web
INFO[0001] [0/1] [web]: Starting
DEBU[0001] Found service web
DEBU[0001] Comparing hashes for web: old: digest.ServiceHash{Service:"ef6f20e9c2931390034d1561104d5547416b525f", LaunchConfig:"1769b015c601991cff358ef2697a68ac53ad5918", SecondaryLaunchConfigs:map[string]string{}} new: digest.ServiceHash{Service:"ef6f20e9c2931390034d1561104d5547416b525f", LaunchConfig:"1769b015c601991cff358ef2697a68ac53ad5918", SecondaryLaunchConfigs:map[string]string(nil)}
1 Like
3

Completely missed this functionality: http://rancher.com/rancher-adds-support-for-private-docker-registries/

For those who are curious, with Google Container Services’s private repository, you can add a custom repository with the username _json_key and the password being the full JSON body from the docs at https://cloud.google.com/container-registry/docs/auth (see section on ‘Using a JSON key file’ – the password is the entire keyfile.json file as weird as that sounds).

This fixed my current issue, but my question still stands: are there other debugging techniques I could be using?

4

Unfortunately, “Operation failed” is typically due to bad credentials. If there are any credentials in Rancher, it will use them (even if you put in docker hub credentials, it will still use the credentials to pull the public DockerHub images).

We are trying to add a feature which will allow better private registry validation, so that the user would know if the credentials are working beforehand.

1 Like