Ah yes… If you’ve got your own DNS entries listing the host IPs then there’s no way for us to remove them from there. What some people have done before is put something like Amazon ELB as a ‘dumb’ TCP balancer in front of the Rancher one, so it can automatically remove failing IPs.
In the last release we added the Route53 DNS catalog item which can automatically keep a Route53 zone up to date with the active host IPs for each service (i.e. serviceName.stackName.environmentName.yourzone.com), and then you can CNAME that to the actual names you want to balance.
There are some bugs in that release with handling disconnected hosts though which you might run in to, those will be fixed in the next release (0.46).