Metadata service will not start if using names instead of I.P

Hi, I’m running rancher 1.6.5 on RancherOs 1.0.3

Rancher is accessible by name: rancher-server-no-ha:8080

And the agent report they can connect to the server…

INFO: Running Agent Registration Process, CATTLE_URL=rancher-server-no-ha:8080/v1
INFO: Attempting to connect to: rancher-server-no-ha:8080/v1
INFO: rancher-server-no-ha:8080/v1 is accessible
INFO: Inspecting host capabilities
INFO: Boot2Docker: false
INFO: Host writable: true
INFO: Token: xxxxxxxx
INFO: Running registration
INFO: Printing Environment
INFO: ENV: CATTLE_ACCESS_KEY=1B50372518DD2AB1072B
INFO: ENV: CATTLE_AGENT_IP=xxx.xxx.xxx.41
INFO: ENV: CATTLE_HOME=/var/lib/cattle
INFO: ENV: CATTLE_REGISTRATION_ACCESS_KEY=registrationToken
INFO: ENV: CATTLE_REGISTRATION_SECRET_KEY=xxxxxxx
INFO: ENV: CATTLE_SECRET_KEY=xxxxxxx
INFO: ENV: CATTLE_URL=rancher-server-no-ha:8080/v1
INFO: ENV: DETECTED_CATTLE_AGENT_IP=xxx.xxx.xxx.41
INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.2.5
INFO: Launched Rancher Agent: 9376356eb3703e2250ebb961de20d419f896e250fd66af4c9f843335453b0d32

which it looks like DNS is working.

But for whatever reason the meta data service can’t seem to connect…

7/22/2017 9:36:15 PMtime=“2017-07-23T01:36:15Z” level=info msg="Subscribing to events"
7/22/2017 9:36:15 PMtime=“2017-07-23T01:36:15Z” level=fatal msg=“Failed to subscribeGet rancher-server-no-ha:8080/v2-beta: dial tcp: lookup rancher-server-no-ha on xxx.xxx.xxx.28:53: no such host”

If I switch the rancher setting on the front end to the private I.P it works.

Names work fine, but they need to be actually resolvable on a container on the host. xxx.xxx.xxx.28 Is being asked but not answering.

Well thats the thing, look at the logs. Why would the agent say it can connect by name?

INFO: Running Agent Registration Process, CATTLE_URL=rancher-server-no-ha:8080/v1
INFO: Attempting to connect to: rancher-server-no-ha:8080/v1
INFO: rancher-server-no-ha:8080/v1 is accessible

Thats the actual name: rancher-server-no-ha

Also I can ping the agent nodes and the rancher server from each other by name…

The resolve.conf on rancher os
domain xxxxxx.xxxxxx
nameserver xxx.xxx.xxx.28
nameserver 8.8.8.8
nameserver 4.2.2.3
nameserver 8.8.4.4

[rancher@rancher-host-0 ~]$ ping rancher-server-no-ha
PING rancher-server-no-ha (xxx.xxx.xxx.44): 56 data bytes
64 bytes from xxx.xxx.xxx.44: seq=0 ttl=64 time=1.662 ms

Any thoughts on this?