Hi, I have an insecure docker registry (installed via arkade install docker-registry) in a private k3s cluster (installed via k3sup) and I cannot pull images from that from my Deployments.
On the master node I have this config active:
$ cat /etc/rancher/k3s/registries.yaml
mirrors:
"192.168.0.103:5000":
endpoint:
- "http://192.168.0.103:5000"
configs:
"192.168.0.103:5000":
auth:
username: admin
password: xxxxxxxxxxxxxx
192.168.0.103 is one of the workers; on the port 5000 I can successfully log in, push and pull images.
On all machines, I have adapted the docker configuration:
$ cat /etc/docker/daemon.json
{
"insecure-registries" : ["192.168.0.103:5000"]
}
On all nodes, docker info lists the insecure registry:
Insecure Registries:
192.168.0.103:5000
127.0.0.0/8
On the master node, in /var/lib/rancher/k3s/agent/etc/containerd/config.toml I can see that the config from registries.yaml sort of “arrived”:
$ cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml
[plugins.opt]
path = "/var/lib/rancher/k3s/agent/containerd"
[plugins.cri]
stream_server_address = "127.0.0.1"
stream_server_port = "10010"
enable_selinux = false
sandbox_image = "docker.io/rancher/pause:3.1"
[plugins.cri.containerd]
snapshotter = "overlayfs"
[plugins.cri.cni]
bin_dir = "/var/lib/rancher/k3s/data/ad8f0f93ebb9db5c507884fcdec249d73dd348293dac194e01462c57815cca46/bin"
conf_dir = "/var/lib/rancher/k3s/agent/etc/cni/net.d"
[plugins.cri.containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."192.168.0.103:5000"]
endpoint = ["http://192.168.0.103:5000"]
[plugins.cri.registry.configs."192.168.0.103:5000".auth]
username = "admin"
password = "xxxxx"
Unfortunately, kubectl describe deployment xxx doesn’t seem to be impressed by my configuration attempts, although everything has been restarted.
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 6s default-scheduler Successfully assigned mydeployment/mypackage-deployment-56fffb99d-7zqp4 to nuc2
Normal Pulling 5s kubelet Pulling image "192.168.0.103:5000/mypackage:1.0.0"
Warning Failed 5s kubelet Failed to pull image "192.168.0.103:5000/mypackage:1.0.0": rpc error: code = Unknown desc = failed to pull and unpack image "192.168.0.103:5000/mypackage:1.0.0": failed to resolve reference "192.168.0.103:5000/mypackage:1.0.0": failed to do request: Head "https://192.168.0.103:5000/v2/mypackage/manifests/1.0.0": http: server gave HTTP response to HTTPS client
Warning Failed 5s kubelet Error: ErrImagePull
Normal BackOff 4s kubelet Back-off pulling image "192.168.0.103:5000/mypackage:1.0.0"
Warning Failed 4s kubelet Error: ImagePullBackOff
What else can I try?
Thank you in advance.