This is a very specific query. Has anybody had any experience configuring a RabbitMQ cluster running with Rancher? I’m running into a lot of difficulty with getting something running, because of RabbitMQ’s dependency on communicating with nodes via specific nodenames. That doesn’t play well with Rancher’s overlay network.
Perhaps it’s worth me explaining a little more about what it is I’m struggling with. In order for RabbitMQ to be able to cluster nodes across a WAN, each node must identify itself with <nodename>@<hostname>. The hostname Rancher assigns to each container appears to resolve on that particular host, so it’s not currently possible for me to cluster nodes across different hosts.
I’ve not attempted to setup a rabbitmq cluster in rancher (yet), but perhaps io.rancher.container.hostname_override and links could get you closer to a working setup… ?
http://docs.rancher.com/rancher/labels/
Thanks for the tip @kaos — that label is really useful to know. Unfortunately it doesn’t solve my problem though, because the hostname isn’t visible across the wider network.
Maybe you could make the nodes register themselves with the overlay IP as hostname?
Something like:
HOST=$(curl http://rancher-metadata/2015-07-25/self/container/primary_ip)That’s a great suggestion, and it’s one that I’ve tried. The problem I’ve run into is that there doesn’t appear to be a way to define a custom HOSTNAME when starting the RabbitMQ process — it derives the env.HOSTNAME and I can’t see any obvious way of overriding that.
I did think about trying to set the hostname to the overlay IP, but I’m a bit concerned about doing that — it doesn’t feel like a particularly clever thing to be doing!
I created a docker image for running RabbitMQ as a cluster within a Rancher service. It uses the Rancher Metadata Service to allow RabbitMQ to scale, restart and add back after a crash.