Deployment fails with ErrImagePull: server gave HTTP response to HTTPS client

Running Rancher v2.4 on a fresh Ubuntu 20.04 box. I installed it on top of microk8s using helm, so not running Docker explicitly - I mean, Docker was never installed on this Ubuntu box by me, yet it’s clearly running in the background. I have succesfully created my project and namespaces and linked to a Gitlab repo. It all works fine to the point that I push to Gitlab, and build and deployment get triggered and they finish green. However, it gets stuck on updating. Specifically, I get the following error:

ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "127.0.0.1:34465/voodoosms/hermes-app:3": failed to resolve reference "127.0.0.1:34465/bentleysoft/hermes-app:3": failed to do request: Head "https://127.0.0.1:34465/v2/bentleysoft/hermes-app/manifests/3": http: server gave HTTP response to HTTPS client

I know that it’s possible to configure Docker to allow insecure registries with a daemon.json as follows

{ "insecure-registries":["myregistry.example.com:5000"] }

However, I’ve no idea how/where to look at in Rancher?

Any help much appreicated!

are you running a private registry on 127.0.0.1 in gitlab and trying to pull the image from workers using 127.0.0.1?
second problem is that u have to config docker in workers to pull from insecure registry and restart docker service. that will put worker in unready starus.

i didnt notice u mentioned its a fresh install. if u havnt deployed any app yet u can restart docker.

Hi, thank you for the suggestion. Sadly, I as in original post, I am not explicitly running Docker. I wish I knew how to alter its startup params but as it’s spwaned by Rancher I have no idea how to do that…

If you installed rancher on top of microk8s then rancher has nothing to do with the configuration of the container runtime underneath it. It’s just another app running in the cluster.

Modern versions of microk8s use containerd, not docker. Configuring an insecure registry is part of its config.toml.

1 Like

Ah, that’s very helpful, I didn’t know and didn’t think of that. I will confirm here shortly as I’ve come across the same question in a few places! Thank you.

I just want to confirm that adding

[plugins."io.containerd.grpc.v1.cri".registry.mirrors."127.0.0.1:34465"]
        endpoint = ["http://127.0.0.1:34465"]

to /var/snap/microk8s/current/args/containerd-template.toml

did the job. Obviously the port number will differ from project to project.

Many thanks vincent!

Should i do it on the host or in docker container ?
And what then? Need to restart something ?