We ran a few more experiments and have somehow came to the following process:
- Don’t use the LB at all
- Press “+1” on the rancher ui to increase containers (across hosts)
- Container GRPC listens at 0.0.0.0 (all interfaces)
- Container reports its IP (machine IP) to an external registry (outside of rancher)
- Container changes its external facing port (in case there is port conflict within same hosts) inside rancher via the Rancher API. This port is given by the external registry who keeps track of port conflicts.
- We found out that there is a nifty “Edit” button for each container where you can change the external facing port.
Essentially we are creating an LB outside of the rancher environment.
Someone told us we may be able to create the LB inside rancher. Then would we need to know the Rancher API in greater detail to make such internal LB (and thus internal networking) work properly?
Anyone has opinions on this? Appreciate the help!