Hi there,
I have 2 hosts in my rancher set-up with a logstash
stack successfully deployed.
When I add a logspout
stack, the container that gets deployed on the same host (host01) as the logstash-collector
works, the other one, deployed on host02, throws an getsockopt: no route to host
error.
logspout.yml:
logspout:
restart: on-failure
environment:
ROUTE_URIS: logstash+tcp://logstash:5000
external_links:
- logstash/logstash-collector:logstash
volumes:
- /var/run/docker.sock:/var/run/docker.sock
labels:
io.rancher.scheduler.global: 'true'
io.rancher.container.pull_image: always
tty: true
image: bekt/logspout-logstash:latest
stdin_open: true
Logs from logspout_1
container:
# logspout v3.2-dev-custom by gliderlabs
# adapters: raw udp tcp logstash
# options : persist:/mnt/routes
# jobs : http[]:80 pump routes
# routes :
# ADAPTER ADDRESS CONTAINERS SOURCES OPTIONS
# logstash logstash:5000 map[]
Logs from logspout_2
container:
# logspout v3.2-dev-custom by gliderlabs
# adapters: logstash raw udp tcp
# options : persist:/mnt/routes
!! dial tcp 10.42.176.8:5000: getsockopt: no route to host
ping
, telnet
& UDP
work fine from within another container deployed on host02:
$ ping 10.42.176.8
PING 10.42.176.8 (10.42.176.8): 56 data bytes
64 bytes from 10.42.176.8: seq=0 ttl=62 time=0.696 ms
64 bytes from 10.42.176.8: seq=1 ttl=62 time=0.781 ms
64 bytes from 10.42.176.8: seq=2 ttl=62 time=0.776 ms
$ telnet 10.42.176.8 5000
Test message from a different container on host02
Kibana
output:
October 15th 2016, 22:07:53.957 message:Test message from a different container on host02 tags:_jsonparsefailure @version:1 @timestamp:October 15th 2016, 22:07:53.957 host:10.42.219.240 port:43030 _id:AVfJ9T6xJw9hW0ybpNep _type:logs _index:logstash-2016.10.15 _score:
$ nc -uv 10.42.176.8 5000
10.42.176.8 (10.42.176.8:5000) open
UDP test message
Kibana
output:
October 15th 2016, 22:09:12.237 host:10.42.219.240 message:UDP test message tags:_jsonparsefailure @version:1 @timestamp:October 15th 2016, 22:09:12.237 _id:AVfJ_NlUJw9hW0ybpNfi _type:logs _index:logstash-2016.10.15 _score:
I also have a load-balancer working, so Rancherās internal DNS is working fine. Also checked the iptables
chains. Everything seems to be correct.
What am I missing?!
Can anybody please tell me why logspout_2
isnāt able to establish a connection with the logstash-collector
?
Thank you
Adrian
EDIT:
If I manually start the logspout_2
container on host2
with host01ās IP it works!!
[rancher@ip-172-31-46-131 ~]$ docker run --name="logspout_xx" \
> --volume=/var/run/docker.sock:/var/run/docker.sock \
> bekt/logspout-logstash:latest \
> logstash+tcp://172.31.46.130:5000
# logspout v3.2-dev-custom by gliderlabs
# adapters: logstash raw udp tcp
# options : persist:/mnt/routes
# jobs : http[]:80 pump routes
# routes :
# ADAPTER ADDRESS CONTAINERS SOURCES OPTIONS
# logstash+tcp172.31.46.130:5000 map[]