Rancher Custom hosts

Rancher documentation (https://rancher.com/docs/rancher/v1.6/en/hosts/) says that we can add any type of hosts on the rancher server, as long as the system has docker working. (Custom, EC2, Azure etc)

I have my 3 environments set up where I have Ubuntu hosts added.
All of my environments working perfectly, untill a recent issue where I see that in one of my environments healthcheck is stuck in Initializing mode. Because of this I am not able to deploy any stack in that environment.

I wanted to check if this is related to custom hosts. If rancher doesn’t recommend adding custom hosts?

This isn’t going to have anything to do with the kind of hosts specifically; but every host in the environment needs to be able to reach every other one in it for the overlay network to work, and healthchecks are performed over that network. So you probably have something blocking 500+4500/udp on one or more of the hosts.

Thanks vincent!! I am verifying the port mapping, and so far dont see any issue with that , however can you please confirm if the docker version can make any difference. I have docker version installed as 17.05.0-ce.

Also from the infrastructure stacks, the Healthcheck stack seems to be in INITIALIZING state and Schedular stack is Unhealthy

It shouldn’t in theory, but docker edge (2nd digit not a multiple of 3) have never been actually supported.

The health service (which checks other services) also checks itself. Each service is checked by a healthcheck on up to 3 other nodes (not on the one the target container is on, if there are >= 4 nodes). And the main thing health checks do is make network requests across the overlay network.