Cannot pull images via Rancher, but can from commandline

Archive Rancher 1.x
1

If I pull images from the docker commandline on the docker server (using ‘docker pull …’) then this works.

However, if the image is not already pulled, then starting up a container via the Docker GUI fails on the pull. Once I’ve manually pulled it, though, containers are started up. This applies to all containers, including ones like rancher/agent-instance for some reason.

This seems to be related to my adding a personal repository, but why should that disable the default repository? Is there some way to get a personal repository in addition to the standard one?

Thanks in advance for any pointers

2

I’ve seen this as well. All the time. It’s really obvious when you launch a standalone container. When you have a service defined, rancher simply starts a new container after the first one failed (but it triggered a pull, so the subsequent container starts ok).

I thought I’d investigate the issue further before reporting it, as it feels like this ought to either a) happen to everyone, so this ought to be a known issue already, or b) something with my setup causing this, in which case I should be able to find out what it is (and failing that, file a bug report). It simply has not been one of my priorities yet…

3

This sounds suspiciously like https://github.com/rancher/rancher/issues/2281, BUT there was another issue where rancher-agent wasn’t handling good error messaging for docker pull errors that was fixed in v0.42.0.

Are both of you running on our latest (v0.42.0)?

4

Ah, true. I did see that issue, but haven’t checked this since we upgraded to 0.42.0.

5

We’re on the latest version, only installed recently.

The problem only seems to happen if there is a custom repository defined; in this case, it seems to disable the standard ones?

6

@sshipway Just to confirm, do you mean custom registry or a personal registry from GitHub or Quay? I’d like to try and reproduce.

7

This was a personal Github registry I set up for testing; once it was there (empty), I could only install containers from the commandline.

I only tested this briefly, and since then I’ve been concentrating on other areas. However, even if I set up the personal registry incorrectly, I still expected the default to be used as a last resort.

Possibly this is a ‘feature’ whereby defining a registry on an environment implicitly disables the default? This could be a good thing, as otherwise there would be no way to disable the default docker.io registry in (eg) a highly-controlled production environment; however it would be good to have it documented and made clear if so.

8

Can you verify what the error is when starting the service?

I am getting an “Operation failed” error, so just want to check.

Edit: I realized that I put in the incorrect credentials, which is why it failed. Once I corrected my credentials, I was able to pull down public Github images through Rancher.

9

Whenever there are credentials in a registry, we will automatically use the credentials in every docker pull for that registry. So if you add in a DockerHub registry, all docker pull to DockerHub will be using the credentials.

10

Ah, this makes sense. So, if I has a typo in my credentials, this would kill ALL the repositories. I’ll check, but this may well be the reason.