Rancher service not discoverable

1

I want to send logs from one rancher service (e.g. my_service) to another rancher service running the ELK stack with the syslog driver

I am setting up my stack via a docker-compose as follows more or less:

 elk-custom:
    # image: elk-custom
    build:
      context: .
      dockerfile: Dockerfile-elk
    ports:
      - 5601:5601
      - 9200:9200
      - 5044:5044
      - 5151:5151
      - 5152:5152


  my_service:
    image: some_image_from_my_local_registry
    depends_on:
      - elk-custom
    logging:
     driver: syslog
     options:
       syslog-address: "tcp://elk-custom:5151"

However, on the stack dashboard, for my_service I get:

my_service (Expected state running but got error: Error response from daemon: failed to initialize logging driver: dial tcp: lookup elk-custom on 10.0.2.3:53: server misbehaving)

Is there anything additional needed to make the specific logging (elk-custom) service discoverable?

2

Where is the 10.0.2.3:53 address coming from?

3

I have no idea…
I have set up a 3-VM (1 rancher server + 2 rancher agent nodes) environment using vagrant and via the following Vagrantfile:

Vagrant.configure("2") do |config|


config.vm.define "lb", primary: true do |lb|
  lb.vm.box = "ubuntu/xenial64"
  lb.vm.synced_folder ".", "/vagrant", disabled: true
  lb.vm.hostname = "server.node.test.com"
  lb.vm.synced_folder "../somefolder/", "/workspace"
  lb.vm.network :private_network, ip: "192.168.1.10"
  lb.vm.provider "virtualbox" do |vb|
    vb.memory = "4096"
    vb.cpus = "2"
  end
  lb.vm.provision "docker" do |dock|
    dock.run "rancher/server",
    args: "-v '/workspace:/workspace' -p 8080:8080"
  end
end




    (1..2).each do |i|
    config.vm.define "esnode#{i}" do |esnode|
      esnode.vm.box = "ubuntu/xenial64"
      esnode.vm.synced_folder ".", "/vagrant", disabled: true
      esnode.vm.hostname = "client#{i}.node.test.com"
      esnode.vm.network :private_network, ip: "192.168.1.1#{i}"
      # esnode.vm.network "forwarded_port", guest: 80, host: "808#{i}"
      esnode.vm.provider "virtualbox" do |vb|
        vb.memory = "3048"
      end
      esnode.vm.provision "docker" do |docknode|
        # docknode.run "rancher/agent:v1.2.6 --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher http://192.168.1.10:8080/v1/scripts/F9E3884A771CB877B1FD:1483142400000:lZFPk8u8xFEaZP3VpdV94YAmL4w"
      end
  end

end

end
4

Logging is done by the docker daemon and is not pointed at Rancher’s dns server, so you cannot use service names as the logging target.

1 Like
5

Thanks; actually I circumvented this on local docker-compose deployment by using 127.0.0.1 as suggested here.

My question now is whether there are any good / best practices on how to go about this?

e.g. the only way is to define another, say logging stack, set a load balancer in front of it and make the stack I want logs from, point to the port exposed by this lb?