Waiting for handler registration (1/2) error

Rancher 1.x
1

I am trying to run a fresh install or rancher on a VM running CentOS 7 3.10.0-327.28.3.el7.x86_64
I have Docker version 1.12.1

docker run --name rancher -d --restart=always -p 0.0.0.0:8080:8080 rancher/server:latest

The problem is that i keep getting errors and errors on the logs, and the web ui and api work in an intermittent way
i see that “websocket-proxy” and "rancher-compose-executor " keeps crashing and starting again.

I tried with firewalld started and stopped.

I tried running other types of containers on the same ports and everything run all right except for rancher.

This is the log, the last part keeps repeating and repeating a couple of times per second.

has anyone had a similar issue ?

$ docker logs -f rancher

160913 0:51:50 [Note] /usr/sbin/mysqld (mysqld 5.5.49-0ubuntu0.14.04.1) starting as process 25 …
Uptime: 2 Threads: 1 Questions: 2 Slow queries: 0 Opens: 33 Flush tables: 1 Open tables: 26 Queries per second avg: 1.000
Setting up database
Importing schema
CATTLE_AGENT_PACKAGE_AGENT_BINARIES_URL=/usr/share/cattle/artifacts/agent-binaries.tar.gz
CATTLE_AGENT_PACKAGE_CADVISOR_URL=/usr/share/cattle/artifacts/cadvisor.tar.gz
CATTLE_AGENT_PACKAGE_HOST_API_URL=/usr/share/cattle/artifacts/host-api.tar.gz
CATTLE_AGENT_PACKAGE_NODE_AGENT_URL=/usr/share/cattle/artifacts/node-agent.tar.gz
CATTLE_AGENT_PACKAGE_PYTHON_AGENT_URL=/usr/share/cattle/artifacts/python-agent.tar.gz
CATTLE_AGENT_PACKAGE_RANCHER_DNS_URL=/usr/share/cattle/artifacts/rancher-dns.tar.gz
CATTLE_AGENT_PACKAGE_RANCHER_METADATA_URL=/usr/share/cattle/artifacts/rancher-metadata.tar.gz
CATTLE_AGENT_PACKAGE_RANCHER_NET_URL=/usr/share/cattle/artifacts/rancher-net.tar.gz
CATTLE_CATTLE_VERSION=v0.165.8.1
CATTLE_DB_CATTLE_DATABASE=mysql
CATTLE_DB_CATTLE_MYSQL_HOST=localhost
CATTLE_DB_CATTLE_MYSQL_NAME=cattle
CATTLE_DB_CATTLE_MYSQL_PORT=3306
CATTLE_DB_CATTLE_USERNAME=cattle
CATTLE_GRAPHITE_HOST=
CATTLE_GRAPHITE_PORT=
CATTLE_HOME=/var/lib/cattle
CATTLE_HOST_API_PROXY_MODE=embedded
CATTLE_LOGBACK_OUTPUT_GELF_HOST=
CATTLE_LOGBACK_OUTPUT_GELF_PORT=
CATTLE_RANCHER_COMPOSE_VERSION=v0.8.6
CATTLE_RANCHER_SERVER_IMAGE=rancher/server
CATTLE_RANCHER_SERVER_VERSION=v1.1.3
CATTLE_USE_LOCAL_ARTIFACTS=true
DEFAULT_CATTLE_API_UI_INDEX=//releases.rancher.com/ui/1.1.12
DEFAULT_CATTLE_CATALOG_EXECUTE=true
DEFAULT_CATTLE_CATALOG_URL=library=https://github.com/rancher/rancher-catalog.git,community=https://github.com/rancher/community-catalog.git
DEFAULT_CATTLE_COMPOSE_EXECUTOR_EXECUTE=true
DEFAULT_CATTLE_MACHINE_EXECUTE=true
DEFAULT_CATTLE_RANCHER_COMPOSE_DARWIN_URL=https://releases.rancher.com/compose/v0.8.6/rancher-compose-darwin-amd64-v0.8.6.tar.gz
DEFAULT_CATTLE_RANCHER_COMPOSE_LINUX_URL=https://releases.rancher.com/compose/v0.8.6/rancher-compose-linux-amd64-v0.8.6.tar.gz
DEFAULT_CATTLE_RANCHER_COMPOSE_WINDOWS_URL=https://releases.rancher.com/compose/v0.8.6/rancher-compose-windows-386-v0.8.6.zip
00:51:58.743 [main] WARN o.e.jetty.security.SecurityHandler - ServletContext@o.e.j.w.WebAppContext@446c5a4{/,file:/usr/share/cattle/2b0025124d75a78e32e6a4abac36d2cd/,STARTING}{file:/usr/share/cattle/2b0025124d75a78e32e6a4abac36d2cd/} has uncovered http methods for path: /
00:52:01.032 [main] INFO ConsoleStatus - [1/6] [0ms] [1ms] Loading config-bootstrap
00:52:01.349 [main] INFO ConsoleStatus - [2/6] [321ms] [313ms] Loading base-config
00:52:01.947 [main] INFO ConsoleStatus - [3/6] [919ms] [598ms] Loading config
00:52:01.958 [main] INFO ConsoleStatus - [4/6] [930ms] [10ms] Starting config-bootstrap
00:52:02.337 [main] INFO ConsoleStatus - [5/6] [1309ms] [379ms] Starting base-config
00:52:02.337 [main] INFO ConsoleStatus - [6/6] [1309ms] [0ms] Starting config
2016-09-13 00:52:02,793 INFO [main] [ConsoleStatus] [1/32] [0ms] [0ms] Loading bootstrap
2016-09-13 00:52:02,954 INFO [main] [ConsoleStatus] [2/32] [161ms] [161ms] Loading config-defaults
2016-09-13 00:52:10,336 WARN [main] [liquibase] modifyDataType will lose primary key/autoincrement/not null settings for mysql. Use and re-specify all configuration if this is the case
2016-09-13 00:52:18,919 INFO [main] [ConsoleStatus] [3/32] [16126ms] [15965ms] Loading system
2016-09-13 00:52:19,003 INFO [main] [ConsoleStatus] [4/32] [16210ms] [84ms] Loading defaults
2016-09-13 00:52:19,309 INFO [main] [ConsoleStatus] [5/32] [16516ms] [306ms] Loading types
2016-09-13 00:52:25,212 INFO [main] [ConsoleStatus] [6/32] [22419ms] [5903ms] Loading system-services
2016-09-13 00:52:25,504 INFO [main] [ConsoleStatus] [7/32] [22710ms] [291ms] Loading agent-server
2016-09-13 00:52:25,817 INFO [main] [ConsoleStatus] [8/32] [23024ms] [313ms] Loading allocator-server
2016-09-13 00:52:27,493 INFO [main] [ConsoleStatus] [9/32] [24700ms] [1676ms] Loading api-server
2016-09-13 00:52:30,304 INFO [main] [ConsoleStatus] [10/32] [27511ms] [2811ms] Loading iaas-api
2016-09-13 00:52:30,395 INFO [main] [ConsoleStatus] [11/32] [27602ms] [91ms] Loading archaius
2016-09-13 00:52:30,674 INFO [main] [ConsoleStatus] [12/32] [27881ms] [279ms] Loading core-model
2016-09-13 00:52:30,740 INFO [main] [ConsoleStatus] [13/32] [27946ms] [65ms] Loading core-object-defaults
2016-09-13 00:52:30,801 INFO [main] [ConsoleStatus] [14/32] [28008ms] [61ms] Loading encryption
2016-09-13 00:52:34,395 INFO [main] [ConsoleStatus] [15/32] [31602ms] [3594ms] Loading process
2016-09-13 00:52:34,455 INFO [main] [ConsoleStatus] [16/32] [31662ms] [60ms] Loading redis
2016-09-13 00:52:34,556 INFO [main] [ConsoleStatus] [17/32] [31763ms] [101ms] Starting bootstrap
2016-09-13 00:52:34,556 INFO [main] [ConsoleStatus] [18/32] [31763ms] [0ms] Starting config-defaults
2016-09-13 00:52:34,556 INFO [main] [ConsoleStatus] [19/32] [31763ms] [0ms] Starting system
2016-09-13 00:52:34,557 INFO [main] [ConsoleStatus] [20/32] [31764ms] [0ms] Starting defaults
2016-09-13 00:52:34,557 INFO [main] [ConsoleStatus] [21/32] [31764ms] [0ms] Starting types
2016-09-13 00:52:40,148 INFO [main] [ConsoleStatus] [22/32] [37355ms] [5591ms] Starting system-services
2016-09-13 00:52:40,149 INFO [main] [ConsoleStatus] [23/32] [37356ms] [1ms] Starting agent-server
2016-09-13 00:52:40,149 INFO [main] [ConsoleStatus] [24/32] [37356ms] [0ms] Starting allocator-server
2016-09-13 00:52:41,459 INFO [main] [ConsoleStatus] [25/32] [38666ms] [1310ms] Starting api-server
time=“2016-09-13T00:52:43Z” level=info msg="Starting websocket proxy. Listening on [:8080], Proxying to cattle API at [localhost:8081], Monitoring parent pid [9]."
time=“2016-09-13T00:52:43Z” level=info msg=“Exiting proxy.” error="Get http://localhost:8081/v1/schemas: dial tcp 127.0.0.1:8081: getsockopt: connection refused"
2016-09-13 00:52:44,263 INFO [main] [ConsoleStatus] [26/32] [41470ms] [2804ms] Starting iaas-api
2016-09-13 00:52:44,263 INFO [main] [ConsoleStatus] [27/32] [41470ms] [0ms] Starting archaius
2016-09-13 00:52:44,264 INFO [main] [ConsoleStatus] [28/32] [41470ms] [0ms] Starting core-model
2016-09-13 00:52:44,264 INFO [main] [ConsoleStatus] [29/32] [41471ms] [0ms] Starting core-object-defaults
2016-09-13 00:52:44,264 INFO [main] [ConsoleStatus] [30/32] [41471ms] [0ms] Starting encryption
2016-09-13 00:52:44,274 INFO [main] [ConsoleStatus] [31/32] [41481ms] [10ms] Starting process
2016-09-13 00:52:44,275 INFO [main] [ConsoleStatus] [32/32] [41482ms] [1ms] Starting redis
00:52:44.440 [main] INFO ConsoleStatus - [DONE ] [48124ms] Startup Succeeded, Listening on port 8081
time=“2016-09-13T00:52:45Z” level=info msg="Starting websocket proxy. Listening on [:8080], Proxying to cattle API at [localhost:8081], Monitoring parent pid [9]."
time=“2016-09-13T00:52:46Z” level=info msg="Downloading certificate from http://localhost:8081/v1/credentials/1c1/certificate"
time=“2016-09-13T00:52:46Z” level=info msg="Starting Rancher Catalog service"
time=“2016-09-13T00:52:46Z” level=info msg="Using catalog library=https://github.com/rancher/rancher-catalog.git"
time=“2016-09-13T00:52:46Z” level=info msg="Using catalog community=https://github.com/rancher/community-catalog.git"
time=“2016-09-13T00:52:46Z” level=info msg=“Setting log level” logLevel=info
time=“2016-09-13T00:52:46Z” level=info msg=“Starting go-machine-service…” gitcommit=v0.31.7
time=“2016-09-13T00:52:46Z” level=info msg="Waiting for handler registration (1/2)"
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x63 pc=0x7fa240626b2c]

runtime stack:
runtime.throw(0xc1b840, 0x2a)
/usr/local/go/src/runtime/panic.go:547 +0x90
runtime.sigpanic()
/usr/local/go/src/runtime/sigpanic_unix.go:12 +0x5a

goroutine 13 [syscall, locked to thread]:
runtime.cgocall(0x8665c0, 0xc82003dbd8, 0x0)
/usr/local/go/src/runtime/cgocall.go:123 +0x11b fp=0xc82003db78 sp=0xc82003db48
net._C2func_getaddrinfo(0x7fa2340008c0, 0x0, 0xc820127530, 0xc82002c500, 0x0, 0x0, 0x0)
??:0 +0x55 fp=0xc82003dbd8 sp=0xc82003db78
net.cgoLookupIPCNAME(0xc820010012, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc87500)
/usr/local/go/src/net/cgo_unix.go:111 +0x448 fp=0xc82003dd50 sp=0xc82003dbd8
net.cgoLookupIP(0xc820010012, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x40e587)
/usr/local/go/src/net/cgo_unix.go:163 +0x56 fp=0xc82003dda8 sp=0xc82003dd50
net.lookupIP(0xc820010012, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/lookup_unix.go:67 +0x94 fp=0xc82003de18 sp=0xc82003dda8
net.glob.func16(0xc875e0, 0xc820010012, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/hook.go:10 +0x4d fp=0xc82003de58 sp=0xc82003de18
net.lookupIPDeadline.func1(0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/lookup.go:106 +0x71 fp=0xc82003ded8 sp=0xc82003de58
internal/singleflight.(*Group).doCall(0x10d8cf0, 0xc820019400, 0xc820010012, 0x9, 0xc820132ae0)
/usr/local/go/src/internal/singleflight/singleflight.go:93 +0x2c fp=0xc82003df88 sp=0xc82003ded8
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc82003df90 sp=0xc82003df88
created by internal/singleflight.(*Group).DoChan
/usr/local/go/src/internal/singleflight/singleflight.go:86 +0x3ee

goroutine 1 [chan receive]:
main.main()
/go/src/github.com/rancher/go-machine-service/main.go:74 +0x38e

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 5 [chan receive]:
github.com/rancher/go-machine-service/vendor/github.com/rancher/event-subscriber/locks.locker()
/go/src/github.com/rancher/go-machine-service/vendor/github.com/rancher/event-subscriber/locks/locks.go:61 +0xdf
created by github.com/rancher/go-machine-service/vendor/github.com/rancher/event-subscriber/locks.init.1
/go/src/github.com/rancher/go-machine-service/vendor/github.com/rancher/event-subscriber/locks/locks.go:7 +0x59

goroutine 7 [syscall]:
os/signal.signal_recv(0x6)
/usr/local/go/src/runtime/sigqueue.go:116 +0x132
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 8 [select]:
net/http.(*persistConn).roundTrip(0xc8201441a0, 0xc820150040, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:1473 +0xf1f
net/http.(*Transport).RoundTrip(0xc8200d2000, 0xc82014c0e0, 0xc8200d2000, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:324 +0x9bb
net/http.send(0xc82014c000, 0x7fa242079578, 0xc8200d2000, 0xecf6944fc, 0x12bbad6b, 0x10d9aa0, 0xc820156000, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:260 +0x6b7
net/http.(*Client).send(0xc820142060, 0xc82014c000, 0xecf6944fc, 0x12bbad6b, 0x10d9aa0, 0xc820152000, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:155 +0x185
net/http.(*Client).doFollowingRedirects(0xc820142060, 0xc82014c000, 0xc87810, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:475 +0x8a4
net/http.(*Client).Do(0xc820142060, 0xc82014c000, 0x14, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:188 +0xff
github.com/rancher/go-machine-service/vendor/github.com/rancher/go-rancher/client.setupRancherBaseClient(0xc820146000, 0xc82011c040, 0x0, 0x0)
/go/src/github.com/rancher/go-machine-service/vendor/github.com/rancher/go-rancher/client/common.go:129 +0x188
github.com/rancher/go-machine-service/vendor/github.com/rancher/go-rancher/client.NewRancherClient(0xc82011c040, 0xc82011c040, 0x0, 0x0)
/go/src/github.com/rancher/go-machine-service/vendor/github.com/rancher/go-rancher/client/generated_client.go:274 +0x4b
github.com/rancher/go-machine-service/vendor/github.com/rancher/event-subscriber/events.NewEventRouter(0xbdcfe0, 0x18, 0x7d0, 0xc82001000b, 0x18, 0xc8200101f2, 0x14, 0xc820016012, 0x28, 0x0, …)
/go/src/github.com/rancher/go-machine-service/vendor/github.com/rancher/event-subscriber/events/listener.go:49 +0xe7
main.main.func1(0xc82001000b, 0x18, 0xc8200101f2, 0x14, 0xc820016012, 0x28, 0xc820134ee0, 0xc820012ae0)
/go/src/github.com/rancher/go-machine-service/main.go:41 +0x360
created by main.main
/go/src/github.com/rancher/go-machine-service/main.go:46 +0x2bf

goroutine 9 [select]:
net/http.(*Transport).getConn(0xc8200d2000, 0xc82011a2a0, 0x0, 0xc82001000b, 0x4, 0xc820010012, 0xe, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:711 +0x4ef
net/http.(*Transport).RoundTrip(0xc8200d2000, 0xc82011a2a0, 0xc8200d2000, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:311 +0x7e9
net/http.send(0xc82011a1c0, 0x7fa242079578, 0xc8200d2000, 0xecf6944fc, 0x12d323a7, 0x10d9aa0, 0xc820132a00, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:260 +0x6b7
net/http.(*Client).send(0xc820127410, 0xc82011a1c0, 0xecf6944fc, 0x12d323a7, 0x10d9aa0, 0xc820088dc0, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:155 +0x185
net/http.(*Client).doFollowingRedirects(0xc820127410, 0xc82011a1c0, 0xc87810, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:475 +0x8a4
net/http.(*Client).Do(0xc820127410, 0xc82011a1c0, 0x14, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:188 +0xff
github.com/rancher/go-machine-service/vendor/github.com/rancher/go-rancher/client.setupRancherBaseClient(0xc82016e000, 0xc820136340, 0x0, 0x0)
/go/src/github.com/rancher/go-machine-service/vendor/github.com/rancher/go-rancher/client/common.go:129 +0x188
github.com/rancher/go-machine-service/vendor/github.com/rancher/go-rancher/client.NewRancherClient(0xc820136340, 0xc820136340, 0x0, 0x0)
/go/src/github.com/rancher/go-machine-service/vendor/github.com/rancher/go-rancher/client/generated_client.go:274 +0x4b
github.com/rancher/go-machine-service/vendor/github.com/rancher/event-subscriber/events.NewEventRouter(0xbc1310, 0x10, 0x7d0, 0xc82001000b, 0x18, 0xc8200101f2, 0x14, 0xc820016012, 0x28, 0x0, …)
/go/src/github.com/rancher/go-machine-service/vendor/github.com/rancher/event-subscriber/events/listener.go:49 +0xe7
main.main.func2(0xc82001000b, 0x18, 0xc8200101f2, 0x14, 0xc820016012, 0x28, 0xc820134ee0, 0xc820012ae0)
/go/src/github.com/rancher/go-machine-service/main.go:57 +0x29e
created by main.main
/go/src/github.com/rancher/go-machine-service/main.go:62 +0x333

goroutine 10 [chan receive]:
main.main.func3(0xc820134ee0)
/go/src/github.com/rancher/go-machine-service/main.go:66 +0x64
created by main.main
/go/src/github.com/rancher/go-machine-service/main.go:72 +0x355

goroutine 19 [select]:
net/http.setRequestCancel.func4(0x0, 0xc8201420c0, 0xc82011c100, 0xc82014e180)
/usr/local/go/src/net/http/client.go:333 +0x12e
created by net/http.setRequestCancel
/usr/local/go/src/net/http/client.go:341 +0x2f7

goroutine 22 [IO wait]:
net.runtime_pollWait(0x7fa240836090, 0x72, 0xc820165000)
/usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc8201540d0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8201540d0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820154070, 0xc820165000, 0x1000, 0x1000, 0x0, 0x7fa242075050, 0xc82000a018)
/usr/local/go/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc820148418, 0xc820165000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7fa240836150, 0xc820148418, 0xc820144208, 0xc820165000, 0x1000, 0x1000, 0x60, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:1687 +0x67
net/http.(*noteEOFReader).Read(0xc8201564a0, 0xc820165000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
:284 +0xd0
bufio.(*Reader).fill(0xc82014e600)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc82014e600, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201441a0)
/usr/local/go/src/net/http/transport.go:1073 +0x177
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:857 +0x10a6

goroutine 11 [runnable]:
net/http.setRequestCancel.func4(0x0, 0xc820127470, 0xc820136400, 0xc820012cc0)
/usr/local/go/src/net/http/client.go:332
created by net/http.setRequestCancel
/usr/local/go/src/net/http/client.go:341 +0x2f7

goroutine 12 [select]:
net.lookupIPDeadline(0xc820010012, 0x9, 0xecf6944fc, 0x12d3cb4d, 0x10d9aa0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/lookup.go:109 +0x6a6
net.internetAddrList(0xb49240, 0x3, 0xc820010012, 0xe, 0xecf6944fc, 0x12d3cb4d, 0x10d9aa0, 0x0, 0x0, 0x0, …)
/usr/local/go/src/net/ipsock.go:252 +0x6ee
net.resolveAddrList(0xb452b0, 0x4, 0xb49240, 0x3, 0xc820010012, 0xe, 0xecf6944fc, 0x12d3cb4d, 0x10d9aa0, 0x0, …)
/usr/local/go/src/net/dial.go:158 +0x466
net.(*Dialer).Dial(0xc8200184b0, 0xb49240, 0x3, 0xc820010012, 0xe, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/dial.go:216 +0x124
net.(*Dialer).Dial-fm(0xb49240, 0x3, 0xc820010012, 0xe, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:38 +0x6e
net/http.(*Transport).dial(0xc8200d2000, 0xb49240, 0x3, 0xc820010012, 0xe, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:659 +0x96
net/http.(*Transport).dialConn(0xc8200d2000, 0x0, 0xc82001000b, 0x4, 0xc820010012, 0xe, 0xbe7b80, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:762 +0x1e3d
net/http.(*Transport).getConn.func4(0xc8200d2000, 0x0, 0xc82001000b, 0x4, 0xc820010012, 0xe, 0xc820012d80)
/usr/local/go/src/net/http/transport.go:706 +0x66
created by net/http.(*Transport).getConn
/usr/local/go/src/net/http/transport.go:708 +0x262

goroutine 23 [runnable]:
fmt.(*pp).doPrintf(0xc820144410, 0xba9c50, 0x10, 0xc820038df8, 0x2, 0x2)
/usr/local/go/src/fmt/print.go:1101
fmt.Fprintf(0x7fa2408361c8, 0xc82011c440, 0xba9c50, 0x10, 0xc820038df8, 0x2, 0x2, 0x1, 0x0, 0x0)
/usr/local/go/src/fmt/print.go:188 +0x74
net/http.(*Request).write(0xc82014c0e0, 0x7fa2408361c8, 0xc82011c440, 0x7fa240836100, 0xc820142660, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/request.go:430 +0x4f7
net/http.(*persistConn).writeLoop(0xc8201441a0)
/usr/local/go/src/net/http/transport.go:1283 +0x2dc
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:858 +0x10cb
time=“2016-09-13T00:52:46Z” level=info msg=“Starting rancher-compose-executor” version=v0.8.6
time=“2016-09-13T00:52:48Z” level=info msg=“Initializing event router” workerCount=10
time=“2016-09-13T00:52:48Z” level=info msg="Connection established"
time=“2016-09-13T00:52:48Z” level=info msg="Starting websocket pings"
time=“2016-09-13T00:52:48Z” level=info msg=“Setting log level” logLevel=info
time=“2016-09-13T00:52:48Z” level=info msg=“Starting go-machine-service…” gitcommit=v0.31.7
time=“2016-09-13T00:52:48Z” level=info msg="Waiting for handler registration (1/2)"
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x63 pc=0x7f2e5c503b2c]

2

exactly the same issue I meet!
except my docker version is 1.10.3
Each time I followed the default step s in installation document to install a fresh latest docker, I got this error and could not open the rancher UI.
But I can install version 1.0.2 without any error if I changed the tag from “stable” to "v1.0.2"
Is there any configuration or any magic I need to cast to get latest version of rancher?

3

Same problem, using either docker 1.10.3, 1.11.2 of 1.12.2 and rancher 1.1.3 or 1.1.4. I can downgrade rancher to 1.0.2 but I’d rather not. I want to run Kubernetes and I then seem have to downgrade docker aswell to 1.10.3.

I’m hitting a brick wall right here. @sail_1 Have you out found what caused it in your environment?

4

Had the, seemingly exact, same problem.

However, I found that my host machine had a resolv.conf that contained a ipv6 address with a zone index (i.e. ipv6 address postfixed by “%3”). This was not being handled correctly by docker, so the container had an illegal /etc/resolv.conf. Once I removed the illegal line, within the Rancher container, the Server came up nicely…

I filed a bug with Docker: https://github.com/docker/docker/issues/28172

5

Hi all, sadly I didn’t find a way to solve this, tried a lot of different stuff and it was a pain that i can make all of this run on AWS but not on my private cloud, same versions of everything.
I tried changing the hypervisor network configurations, disabling IPv6 from the kernel, downgrading versions and a hole lot more. Nothing worked.
And since I was working on a rush project that i needed to demo ASAP my safest and easies bet was to change to ubuntu cloud image 16.04, surprisingly everything ran out of the box without any actual changes from my side.
I feel bad that I just quit the challenge and never came back to it, but the job had to be done in a really short timeframe.
Sorry i can’t be of any more help

6

Same issue here, CentOS 7 7.3.1611, ipv6 disabled.
This is the snip of the error:
time=“2016-12-15T14:16:23Z” level=info msg="Reply: 2bc5ce36-90a5-4109-a986-248a52b6a5e8, compute.instance.activate, 1ihm59:instanceHostMap"
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x63 pc=0x7f6b703a7259]

runtime stack:
runtime.throw(0x8d2ca8, 0x2a)
    /usr/local/go/src/runtime/panic.go:566 +0x95
runtime.sigpanic()
    /usr/local/go/src/runtime/sigpanic_unix.go:12 +0x2cc

goroutine 207 [syscall, locked to thread]:
runtime.cgocall(0x70c400, 0xc42029ec90, 0xc400000000)
    /usr/local/go/src/runtime/cgocall.go:131 +0x110 fp=0xc42029ec48 sp=0xc42029ec08
net._C2func_getaddrinfo(0x7f6b600008c0, 0x0, 0xc420396000, 0xc420028020, 0x0, 0x0, 0x0)
    ??:0 +0x68 fp=0xc42029ec90 sp=0xc42029ec48
net.cgoLookupIPCNAME(0xc4203cc060, 0x14, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/cgo_unix.go:146 +0x37c fp=0xc42029edb0 sp=0xc42029ec90
net.cgoLookupIP(0xcbe340, 0xc42000c238, 0xc4203cc060, 0x14, 0x416a37, 0xc42029efd8, 0x7f6b71809598, 0xc4203cc040, 0x20, 0x0)
    /usr/local/go/src/net/cgo_unix.go:204 +0x3a1 fp=0xc42029ef18 sp=0xc42029edb0
net.lookupIP(0xcbe340, 0xc42000c238, 0xc4203cc060, 0x14, 0x4, 0x0, 0x8, 0xc42029f008, 0xc42029eff0)
    /usr/local/go/src/net/lookup_unix.go:70 +0xf9 fp=0xc42029ef78 sp=0xc42029ef18
net.glob..func11(0xcbe340, 0xc42000c238, 0x902618, 0xc4203cc060, 0x14, 0x409674, 0x7e75e0, 0xc4202d2088, 0xc42029f128, 0xc42029f070)
    /usr/local/go/src/net/hook.go:19 +0x52 fp=0xc42029efd0 sp=0xc42029ef78
net.lookupIPMerge.func1(0x0, 0xc4202b0000, 0x4b2864, 0xc4202b0000)
    /usr/local/go/src/net/lookup.go:69 +0x5f fp=0xc42029f058 sp=0xc42029efd0
internal/singleflight.(*Group).doCall(0xcd4920, 0xc4202b0000, 0xc4203cc060, 0x14, 0xc42029f1e8)
    /usr/local/go/src/internal/singleflight/singleflight.go:93 +0x3c fp=0xc42029f0e8 sp=0xc42029f058
internal/singleflight.(*Group).Do(0xcd4920, 0xc4203cc060, 0x14, 0xc42029f1e8, 0x14, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/internal/singleflight/singleflight.go:63 +0x1b9 fp=0xc42029f150 sp=0xc42029f0e8
net.lookupIPMerge(0xcbe340, 0xc42000c238, 0xc4203cc060, 0x14, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup.go:70 +0xe1 fp=0xc42029f220 sp=0xc42029f150
net.LookupIP(0xc4203cc060, 0x14, 0x0, 0x0, 0xc420326000, 0xcba500, 0xc4203cc000)
    /usr/local/go/src/net/lookup.go:51 +0xb6 fp=0xc42029f338 sp=0xc42029f220
github.com/rancher/agent/vendor/github.com/ShowMax/go-fqdn.Get(0xc420326000, 0xc420490000)
    /go/src/github.com/rancher/agent/vendor/github.com/ShowMax/go-fqdn/fqdn.go:17 +0x58 fp=0xc42029f3c0 sp=0xc42029f338
github.com/rancher/agent/utilities/config.getFQDNLinux(0x2a, 0x0, 0xc4203ca1b0, 0x24)
    /go/src/github.com/rancher/agent/utilities/config/hostname_unix.go:25 +0x8e fp=0xc42029f410 sp=0xc42029f3c0
github.com/rancher/agent/utilities/config.Hostname(0x0, 0xc4203ca1b0, 0x24, 0x0)
    /go/src/github.com/rancher/agent/utilities/config/hostname_unix.go:13 +0x26 fp=0xc42029f450 sp=0xc42029f410
github.com/rancher/agent/utilities/config.PhysicalHost(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/src/github.com/rancher/agent/utilities/config/config.go:158 +0x10b fp=0xc42029f5a8 sp=0xc42029f450
github.com/rancher/agent/core/ping.addResource(0xc420330000, 0xc42029fc40, 0xc42028aa10, 0xc4202ba0f0, 0x5, 0x5, 0x8c3b01, 0x4)
    /go/src/github.com/rancher/agent/core/ping/ping_common.go:34 +0xd6 fp=0xc42029fbb0 sp=0xc42029f5a8
github.com/rancher/agent/core/ping.DoPingAction(0xc420330000, 0xc42029fc40, 0xc42028aa10, 0xc4202ba0f0, 0x5, 0x5, 0xc4200214e8, 0x406795)
    /go/src/github.com/rancher/agent/core/ping/ping.go:17 +0x72 fp=0xc42029fc00 sp=0xc42029fbb0
github.com/rancher/agent/handlers.(*PingHandler).Ping(0xc420328140, 0xc420330000, 0xc42033e000, 0x1, 0xc420021520)
    /go/src/github.com/rancher/agent/handlers/ping.go:29 +0xe5 fp=0xc42029fc90 sp=0xc42029fc00
github.com/rancher/agent/handlers.(*PingHandler).Ping-fm(0xc420330000, 0xc42033e000, 0xc420021570, 0x5e1101)
    /go/src/github.com/rancher/agent/handlers/common.go:41 +0x3e fp=0xc42029fcc8 sp=0xc42029fc90
github.com/rancher/agent/handlers.cleanLog.func1(0xc420330000, 0xc42033e000, 0xc4201b0018, 0x4)
    /go/src/github.com/rancher/agent/handlers/common.go:55 +0x59 fp=0xc42029fd60 sp=0xc42029fcc8
github.com/rancher/agent/vendor/github.com/rancher/event-subscriber/events.doWork(0xc420330000, 0xc420385f20, 0xc42033e000, 0xcb9ac0, 0xc4201b0150)
    /go/src/github.com/rancher/agent/vendor/github.com/rancher/event-subscriber/events/worker.go:94 +0x2a4 fp=0xc42029ff30 sp=0xc42029fd60
github.com/rancher/agent/vendor/github.com/rancher/event-subscriber/events.(*skippingWorkerPool).HandleWork.func1(0xc420383350, 0x2b, 0xc420330000, 0xc420385f20, 0xc42033e000)
    /go/src/github.com/rancher/agent/vendor/github.com/rancher/event-subscriber/events/worker.go:47 +0x9e fp=0xc42029ff68 sp=0xc42029ff30
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2086 +0x1 fp=0xc42029ff70 sp=0xc42029ff68
created by github.com/rancher/agent/vendor/github.com/rancher/event-subscriber/events.(*skippingWorkerPool).HandleWork
    /go/src/github.com/rancher/agent/vendor/github.com/rancher/event-subscriber/events/worker.go:48 +0x19f