Issues communicating across hosts

Rancher 1.x
1

I have the current host setup

host1

  • testproject

host2

  • consul.discovery

When I ping consul.discovery from testproject, I can ping the ip and communicates, but cannot curl against it, however if I moved testproject to host2, it can work fine

they are both in AWS, same VPC, same security groups (opens all traffic locally)

Any ideas?

– tried verifying port open, here is result from nmap
bash-4.3# nmap -p 8500 consul.discovery

Starting Nmap 7.40 ( https://nmap.org ) at 2017-07-13 17:26 UTC
Nmap scan report for consul.discovery (10.42.137.51)
Host is up (0.0034s latency).
rDNS record for 10.42.137.51: ip-10-42-137-51.ec2.internal
PORT STATE SERVICE
8500/tcp filtered fmtp
MAC Address: 02:FD:D8:37:5B:E2 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 0.65 seconds
bash-4.3#

2

Looks like one of the hosts was using a public ip and other was private, once i readded both with private ip, it works now