I’ve been running RancherOS for about 18 months now and it has been flawless. However, earlier this week I ran into a problem building new stacks. The specific error is thrown by
Container should have been running but is in error state. Check logs for more information.: Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
In doing some research, I’ve tried confirming that the DNS services look right:
[rancher@ArmstrongLabs ~]$ cat /etc/resolv.conf nameserver 188.8.131.52 nameserver 184.108.40.206 nameserver 192.168.2.1
and I’m able to ping 220.127.116.11
[rancher@ArmstrongLabs ~]$ ping 18.104.22.168 PING 22.214.171.124 (126.96.36.199): 56 data bytes 64 bytes from 188.8.131.52: seq=0 ttl=119 time=21.476 ms 64 bytes from 184.108.40.206: seq=1 ttl=119 time=21.956 ms 64 bytes from 220.127.116.11: seq=2 ttl=119 time=21.755 ms
I did note that I’m able to
ping 18.104.22.168 and
192.168.2.1. The the docker index looks correct (?):
[rancher@ArmstrongLabs ~]$ nslookup index.docker.io Server: 22.214.171.124 Address 1: 126.96.36.199 dns.google Name: index.docker.io Address 1: 188.8.131.52 ec2-50-16-172-3.compute-1.amazonaws.com Address 2: 184.108.40.206 ec2-52-3-104-67.compute-1.amazonaws.com Address 3: 220.127.116.11 ec2-34-193-164-221.compute-1.amazonaws.com Address 4: 18.104.22.168 ec2-3-220-75-233.compute-1.amazonaws.com Address 5: 22.214.171.124 ec2-54-236-224-26.compute-1.amazonaws.com Address 6: 126.96.36.199 ec2-54-81-26-184.compute-1.amazonaws.com
To ensure I’ve isolated what I think the problem is – I’ve built a very simple MQTT using
version: '2' services: MQTT: image: eclipse-mosquitto:1.6.8 #volumes: #- mqttBrokerStore:/mosquitto ports: - 1883:1883/tcp - 9001:9001/tcp
Version 1.6.8 sits on my server. When I change the image to
:latest so that it would force a new pull, the error above is generated.
To double check that it isn’t a router issue – I performed all the above from a docker instance running on my PC without issues or errors.
Out of curiosity, I followed the
https://registry-1.docker.io/v2/ link and it does return an error that says
"authentication required". I don’t think this is the issue - the error I’m getting in RancherOS isn’t a 401/403 error type (?).
RancherOS was a bare metal install on a Rizen 9.
[rancher@ArmstrongLabs ~]$ sudo ros os version v1.5.6
- As part of my trouble shooting I upgraded from v.1.5.4 to v.1.5.6 – no effect.
Any thoughts or directions to troubleshoot further would be greatly appreciated.