rancherOS 0.4.3 registry issue

Archive RancherOS
1

Hi,
I upgraded to rancherOS 0.4.3 today and now it fails to pull images from another rancher 0.4.3 running registry:2 (2.3.0)

It ends with:
Error response from daemon: Get http://mydockerhub:5000/v2/monitoring/sensu-client/manifests/latest: read tcp 10.215.154.106:41762->10.215.154.181:5000: read: connection reset by peer

Is this a known issue or should I start digging down into it?

br hw

2

some more info:

time=“2016-02-21T08:06:44.908455874Z” level=debug msg=“Calling POST /v1.22/images/create”
time=“2016-02-21T08:06:44.908642289Z” level=debug msg=“POST /v1.22/images/create?fromImage=mydockerhub%3A5000%2Fmonitoring%2Fsensu-client%3Alatest”
time=“2016-02-21T08:06:44.909027481Z” level=debug msg=“Trying to pull mydockerhub:5000/monitoring/sensu-client from https://mydockerhub:5000 v2”
time=“2016-02-21T08:06:44.916388769Z” level=warning msg=“Error getting v2 registry: Get https://mydockerhub:5000/v2/: tls: oversized record received with length 20527”
time=“2016-02-21T08:06:44.916644462Z” level=debug msg=“Trying to pull mydockerhub.local:5000/monitoring/sensu-client from http://mydockerhub:5000 v2”
time=“2016-02-21T08:08:45.223271077Z” level=debug msg=“Error trying v2 registry: Get http://mydockerhub:5000/v2/monitoring/sensu-client/manifests/latest: read tcp 10.215.154.113:44940->10.215.154.181:5000: read: connection reset by peer”
time=“2016-02-21T08:08:45.223738576Z” level=debug msg=“Skipping v1 endpoint https://mydockerhub:5000 because v2 registry was detected”
time=“2016-02-21T08:08:45.223785696Z” level=debug msg=“Skipping v1 endpoint http://mydockerhub:5000 because v2 registry was detected”
time=“2016-02-21T08:08:45.223911338Z” level=error msg=“Handler for POST /v1.22/images/create returned error: Get http://mydockerhub:5000/v2/monitoring/sensu-client/manifests/latest: read tcp 10.215.154.113:44940->10.215.154.181:5000: read: connection reset by peer”

Right now I cannot use 0.4.3 with private registry so I had to revert back to 0.4.1/0.4.2.

br hw

3

Are you using RancherOS with Rancher? I just saw someone else have the same issue “connection reset by peer” with a different operating system (CentOS 7)

Is this happening with any registry pull or just within Rancher?

4

Hi,
I’ve added an --insecure-registry to RancherOS and pulling from shell.

I’ve noticed the following:

Scenario:

  • Running a nexus 3 repository allowing also V1 client API (it’s running on another host in the OpenStack cluster, managed by Rancher)
  • making a simple docker project (from debian:jessie, RUN echo ‘Hi’)
  1. building an image with docker 1.10.3 => 0.4.3 will not pull image
  2. Building an image with RancherOS 0.4.2 (1.9.1) => 0.4.3 will not download image
  3. Building an image with RancherOS 0.4.3 push OK, delete local images, pull => will not download image (read: connection reset by peer)
  4. push 1.10.3 image, RancherOS 0.4.2 cannot find image

I’m running OpenStack and will be helpful testing out an dev 0.4.4 openstack image if you can provide one.

/hw

5

In docker 1.10, Docker changed the docker image files. Could it be related to this change? What version of docker is running on your 0.4.2 servers?

6

@Rucknar,
yeah thats probably the reason for the issue with ROS 0.4.2 (docker 1.9.1)

But the pull issue on 0.4.3 must be something else.

/hw

7

@hwaastad We recently released v0.4.4 and an openstack image with it. Could you test it and see if you are still having issues?

There are still some known registry support requests for when you want to pull in a private image through cloud-init that’s not supported.

8

Hi,
the issues I have are partially been resolved.
Thst is, it is due to the mtu challenges in docker >=1.10 and RancherOS.

Br hw