Cross Container and Host Network Troubleshooting

I am trying to solve the issue between Datacenters and AWS. In each datacenter we have few rancher hosts as well as few hosts in AWS. Datacenters connectivity is pretty much wide open to AWS through the VPN.

I can access the container from all DCs to AWS with public dns record however I am trying to ‘avoid’ that if possible.
Almost all hosts are coupled together in one UI per environment

DC-Dev1, DC-Dev2, AWS-DEV1

I am able to reach AWSDEV1(env) from containerA to containerB by just doing like http://containerA:port
I am able to reach AWSDEV1(env) from containerA to containerA by doing like http://container.stack.env.internal
I am able to reach from DC1 or DC2/3/4 to AWSDEV1 with public IP which is secured by SG on aws so that only DC IP’s can hit that url.

What I am NOT able to do:
Reach from say DC1 to AWSDev1 container unless its a publicly available domains

So how do I configure my DC/AWS hosts for containers to talk to each other?