I’m considering Rancher for project, but I’m wondering how to solve one networking issue:
For flawless managed network there is need to have direct visibility of each node (if i understand correctly, haven’t tested it yet). This could be quite an issue when you don’t want to put compute nodes visible (this is something you don’t want to do in private container service.
For scenario where you have all infrastructure in-house, you can allow them to communicate directly, but adding AWS infrastructure will deny the possibility to use managed network. Possibility would be to use P2P VPN network (such as PeerVPN / n2n / …, depends on flexibility / performance).
Have you considered implementing P2P network between compute nodes? If not, what would be the best place to start implementing it? (I know Rancher utilizes Racoon, but that are just the basics)