Hello,
I am struggling to fix the issue related to catalog, it sometimes fail with following error, not all time. Can someone give a clue? Just want to know what is wrong! I have 20 containers in single catalog, is it causing issue? or some issues related to rancher or registry? or even haproxy infront of rancher?
Error (Get http://localhost:8080/v2-beta/projects/1a87277/services/1s1450: net/http: request canceled (Client.Timeout exceeded while awaiting headers))
Please help
Tried with just 3 containers docker-compose.yml plus keeping all those variables put in rancher-compose(around 150 variables with 1000 lines), it fails there again. So its not related to number of containers, is it anyway related to number of lines in rancher-compose?
Though containers in it are running fine, its showing as stack is invalid. So I cant clone or add another service to the same catalog, getting error like " Validation failed in API: StackId"
Since its HA mode behind an haproxy, I restarted the slave container and I am getting the similar kind of error there too. It started happening after I upgraded the rancher server to 1.6.5
_> 12:07:01.208 [main] INFO ConsoleStatus - Cluster membership changed [:9345, :9345] _
_> 12:07:01.210 [main] INFO ConsoleStatus - Checking cluster state on start-up _
_> time=“2017-08-25T12:07:23Z” level=info msg=“Downloading key from http://localhost:8081/v1/scripts/api.crt” _
_> time=“2017-08-25T12:07:23Z” level=fatal msg=“Error getting config.” error=“Invalid key content” _
_> time=“2017-08-25T12:07:25Z” level=info msg=“Downloading key from http://localhost:8081/v1/scripts/api.crt” _
_> time=“2017-08-25T12:07:25Z” level=fatal msg=“Error getting config.” error=“Invalid key content” _
_> time=“2017-08-25T12:07:27Z” level=info msg=“Downloading key from http://localhost:8081/v1/scripts/api.crt” _
> time=“2017-08-25T12:07:28Z” level=info msg="Starting websocket proxy. Listening on [:8080], Proxying to cattle API at [localhost:8081], Monitorin
_> g parent pid [9]." _
_> time=“2017-08-25T12:07:28Z” level=info msg=“Configured http API filter” _
_> time=“2017-08-25T12:07:28Z” level=info msg=“Master config file: master.conf” _
_> time=“2017-08-25T12:07:28Z” level=info msg=“Master address: IPADDRESS2:8080” _
_> time=“2017-08-25T12:07:28Z” level=info msg=“Configured authTokenValidator API filter” _
_> time=“2017-08-25T12:07:28Z” level=info msg=“Downloading certificate from http://localhost:8081/v1/credentials/1c1/certificate” _
_> time=“2017-08-25T12:07:29Z” level=info msg=“Webhook service listening on 8085” _
_> time=“2017-08-25T12:07:36Z” level=info msg=“Starting Rancher Auth service” _
_> time=“2017-08-25T12:07:36Z” level=info msg=“Listening on :8090” _
time=“2017-08-25T12:07:41Z” level=fatal msg="Get http://localhost:8080/v2-beta: net/http: request canceled (Client.Timeout exceeded while awaiting headers) _
_> 2017/08/25 12:07:41 http: proxy error: net/http: request canceled _
> time=“2017-08-25T12:07:52Z” level=info msg=“Starting Catalog Service (port 8088, refresh interval 300 seconds)”