Can't get Rancher back up

Rancher 1.x
1

i had Rancher previously installed and working. I setup a couple environments and some stacks inside them. then suddenly my Rancher access stopped working today, so i rebooted the server and couldn’t get it to come back up. I am using an external DB with the following command to run the server:

sudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server
–db-host myhost.example.com --db-port 3306 --db-user username --db-pass password --db-name cattle

but the logs show its stuck at this:

2017-08-06 18:03:12,830 INFO [main] [ConsoleStatus] Starting DB migration
2017-08-06 18:03:15,292 INFO [main] [ConsoleStatus] DB migration done
2017-08-06 18:03:15,879 INFO [main] [ConsoleStatus] Cluster membership changed [127.0.0.1:9345]
2017-08-06 18:03:15,881 INFO [main] [ConsoleStatus] Checking cluster state on start-up
2017-08-06 18:03:15,881 INFO [main] [ConsoleStatus] Waiting to become master

it’s been there for several hours now and i’ve tried multiple times. any idea how to fix this?

2

In case anyone runs into this, I was able to fix it by going into the external Rancher DB. It appears that when I was running the docker run... command, it was creating 2 records in the cluster_membership table. I just deleted the first record that it created, then it was able to finish getting up and running after that.

3

I have similiar problem now with rancher server 1.6.7. I have two servers connected HA with shared database. I started third now and this stopped on “INFO [main] [ConsoleStatus] Waiting to become master”. It does not matter how many more I run, all this new servers will stop at “Waiting to become master”.

4

I found the problem with my setup:

I started servers via docker-compose and there:

This is good:

command: --db-host sqlserver --db-port 3306 --db-user root --db-pass abcd1234 --db-name rancher --advertise-address myserver1:9345 --advertise-http-port 443

This is bad:

 environment:
    - CATTLE_DB_CATTLE_MYSQL_HOST=sqlserver
    - CATTLE_DB_CATTLE_MYSQL_PORT=3306
    - CATTLE_DB_CATTLE_MYSQL_NAME=rancher
    - CATTLE_DB_CATTLE_USERNAME=root
    - CATTLE_DB_CATTLE_PASSWORD=abcd1234
    - CATTLE_CLUSTER_ADVERTISE_ADDRESS=myserver1:9345
    - CATTLE_CLUSTER_ADVERTISE_HTTP_PORT=443

!!