503 after server restart

Rancher 1.x
1

I restarted my docker environment cause i updated my docker version.
After that I’m no longer able to connect to the console or logs. It dose not mater weather I Use rancher-compose or the WebUI.

Here the output of the rancher command:

ERRO[0002] Failed to get logs for Webserver_redis_1: websocket: bad handshake
ERRO[0002] Failed to get logs for Webserver_ShiftheroWeb_1: websocket: bad handshake
ERRO[0002] Failed to get logs for Webserver_GitlabCiRunner_1: websocket: bad handshake
ERRO[0002] Failed to get logs for Webserver_Gitlab_1: websocket: bad handshake

Any suggestions?

Greetings from Switzerland
Thomas

2

Can you please provide your docker version and Rancher version?

Also, could you get more details on your setup? Is it a single node or multi-node? Do you have any special configuration with it?

3

Finally I was able to solve the problem by launching a new instance of the rancher/server and network agent container.

A Restart of the rancher containers didn’t help

My environment is:

Single node with external DB
Rancher	v0.39.0
Cattle	v0.95.0
User Interface	v0.52.0
Rancher Compose	beta/latest
Docker version 1.7.1, build 786b29d

No special config

4

Do you still have the broken Rancher server container running? Wondering if we could get the logs to see what was reporting on that side after you restarted.

5

Are you on AWS by any chance? You could have ephemeral IPs that changed when the server was restarted.

6

@vincent I’m on AWS but i haven’t rebooted the box and I use a fix IP

@denise I have started a new instance of the same image to get it running. The old instance is Unfortunately already deleted. As this is my production environment I would not like to restart the current server instance to reproduce the issue.

7

@zauberertz No worries on restarting the broken server. I was just hoping to get some logs. Next time, could you try and grab some logs so that we could troubleshoot your issue?
http://docs.rancher.com/rancher/faqs/troubleshooting/#server-logs