Unable to Pull Docker Images (v.1.5.6)

I’ve been running RancherOS for about 18 months now and it has been flawless. However, earlier this week I ran into a problem building new stacks. The specific error is thrown by service.upgradewait.exception:

Container should have been running but is in error state. Check logs for more information.: Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

In doing some research, I’ve tried confirming that the DNS services look right:

[rancher@ArmstrongLabs ~]$ cat /etc/resolv.conf

and I’m able to ping

[rancher@ArmstrongLabs ~]$ ping
PING ( 56 data bytes
64 bytes from seq=0 ttl=119 time=21.476 ms
64 bytes from seq=1 ttl=119 time=21.956 ms
64 bytes from seq=2 ttl=119 time=21.755 ms

I did note that I’m able to ping and The the docker index looks correct (?):

[rancher@ArmstrongLabs ~]$ nslookup index.docker.io
Address 1: dns.google

Name:      index.docker.io
Address 1: ec2-50-16-172-3.compute-1.amazonaws.com
Address 2: ec2-52-3-104-67.compute-1.amazonaws.com
Address 3: ec2-34-193-164-221.compute-1.amazonaws.com
Address 4: ec2-3-220-75-233.compute-1.amazonaws.com
Address 5: ec2-54-236-224-26.compute-1.amazonaws.com
Address 6: ec2-54-81-26-184.compute-1.amazonaws.com

To ensure I’ve isolated what I think the problem is – I’ve built a very simple MQTT using docker-compose.yml:

version: '2'
    image: eclipse-mosquitto:1.6.8
    #- mqttBrokerStore:/mosquitto
    - 1883:1883/tcp
    - 9001:9001/tcp

Version 1.6.8 sits on my server. When I change the image to :latest so that it would force a new pull, the error above is generated.

To double check that it isn’t a router issue – I performed all the above from a docker instance running on my PC without issues or errors.

Out of curiosity, I followed the https://registry-1.docker.io/v2/ link and it does return an error that says "authentication required". I don’t think this is the issue - the error I’m getting in RancherOS isn’t a 401/403 error type (?).

System Info:

RancherOS was a bare metal install on a Rizen 9.

[rancher@ArmstrongLabs ~]$ sudo ros os version
  • As part of my trouble shooting I upgraded from v.1.5.4 to v.1.5.6 – no effect.

Any thoughts or directions to troubleshoot further would be greatly appreciated.