Update Registry

Rancher 2.x
1

I have a Rancher installed in version 2.4.7 and I just upgrade it in 2.6.3 (in a lab environnement at first).

We have also have a new registry, to replace the docker registry used since before.

After the upgrade some clusters didn’t come up because of ‘ImagePullBackOfff’ issue. I have tried to patch the pods, it’s working some seconds and the cluster get back to the old registry.

So my question is, how to configure the cluster to use the new registry ?

2

Which components or pods are failing due to the ImagePullBackoff? You might need to update the cluster level or global default registries and ensure the images are present there.

3

Here is the pods witht the status ImagePullBackOff.

  • cattle-cluster-agent
  • cattle-node-agent
  • kube-api-auth

I have finally fix the cluster by patching directly the pods and not the deployment or daemonset. Now the cluster is up.

I have reconfigured the registry in the cluster.yaml to set the new registry first with is_default: true followed by the old registry with is_default: false.

Now everything it’s ok.

Now I tried to upgrade my cluster but I have the following error in the Rancher UI
[Error response from daemon: unauthorized: The client does not have permission for manifest]

I don’t know which image Rancher is trying to pull and why it’s doesn’t work?

4

I finaly figured out what it’s going wrong.

The new registry needs to be in the first with is_default: true followed by the oldregistrry 'juste in case the cluster try to pull images form the old one)

private_registries:
- ecr_credential_plugin: {}
is_default: true
url: ‘’
user: xxx
password: xxx
- ecr_credential_plugin: {}
is_default: false
url: ‘’
user: xxx
password: xxx