I just want to make sure that all the traffic that flows between the rancher hosts is encrypted and goes via the IPsec tunnel as default or are there any exception cases?
Moreover, what are the ports that I need to open between the Rancher server and the hosts?
IPSec is for container-to-container traffic. agent-to-server is via the host registration URL. If you want it to be TLS you need to run a balancer or proxy that does TLS-termination and set the registration URL to https.