Hello,
I’am a big fan of rancher since a couple of months and today we are about to go on and switch to a production environment.
BUT
In our microservices architecture we are using consul as discovery catalog and in order to be totally decoupled we are using registrator to automatically register/deregister services into consul. Then, consul-template dynamically update our HAProxy configuration. This works perfectly … in 1.1.x
In 1.2.x, its a completly different story
One of advantages of rancher (imho) is the “managed network” functionnality with the ability to do a DNS load balancing. But, in 1.2.x and “managed network” mode, containers are now deployed with “none” docker networking, so registrator isn’t able to do its job anymore. None of our services are registered now with 1.2.x
This “third party registration” is a very important aspect of our architecture, so, how can we achieve it with the new version of rancher? Does anyboby else is using registrator ?
I am also impacted by this change… pre 1.2 I used this method of service registration without problem, it does not work with the CNI network plugin that’s used > 1.2.
One way to make this work would be to write something (own implementation of registrator) that interfaces with the rancher API to obtain information about services and then register and deregister them in consul…
Someone gives me another interesting way to achieve this by using the metadata service of rancher :
It could be done by looking at rancher-metadata…/container/ports/ will give you x.x.x.x:PORT:internal mappings. Where x.x.x.x will be a specific IP that is bound, or 0.0.0.0 which could be substituted with the ./host/agent_ip. I am using relative paths, because it would depend on the entity responsible for registering. The container would use the self paths and a separate observer would watch the /container or /services endpoint. Here are the metadata docs: http://docs.rancher.com/rancher/v1.3/en/rancher-services/metadata-service/
I think I will create my own “registrator” plugged to the rancher metadata api
I started writing a component in nodejs using the module docker-event which is an event-emitter based module. It allows to react to different events that occur on the docker.sock (like registrator).
Confd has an existing integration with the rancher API. You can use that to easily get the containers and stacks and then write your own hook to push those to consul
That’s an interesting idea @cpoole. We had been using Registrator as well.
We use Consul in our environment outside of Rancher and so we were using registrator to map the public host port from Rancher into Consul so that external services could discover it.
I think the confd solution would be great. Just need a small container with Consul, confd, and monit on it. So that confd can single consul to reload the config files without a full restart start using a SIGHUP.
You can use -useIpFromLabel=io.rancher.container.ip(or requested_ip, I’m not able to check it right now) flag and -internal flag in registrator to get ip and exposed ports. It works fine for me. But this feature only available in registrator’s master branch as I remember.
Hmmm … My problem is that I do not use the same port exposed internally on the host. For example I have containers port mapping like this : 50012:81/tcp
So in this case, -internal will register 81 instead of 50012 no ?
Yes, it’ll register internal port, but, you can use it without -internal flag. And, as far as I understood, you want to get mapped ports with host ip, doesn’t you? So, why don’t you want to use bridge mode with “io.rancher.container.network=true” for interanal networking. If you use it, registrator will take host ip and you needn’t to use -useIpFromLabel flag.
We just resolvd our issue which is pretty similar by running the registrator master image and using the -useIpFromLabel. Registrator picked up the container with exposed ports and properly registered with Consul. We are using VXLAN network in Rancher, we also are able to route directly to containers.
I’d like to thank @carinoob for his work on the rancher-registrator library.
It’s inspiring, and made me do some hacking on his library, that lead to me doing some hacking on the original gliderlabs registrator project.
I’ve done some commits to the original registrator project that get the external port from the rancher-cni using the rancher-metadata project. This works for me - and I’m looking for feedback (so the gliderlabs guys can merge it back into main).
sry for this little off topic question, @cabrinoob can you tell me the name of the tool to make those neat graphics?
I have to make an architechture presentation soon showing a proposal for docker
Thanks!
