Internal Network seems partially split

Any idea what can cause some containers to be inaccessible cross-stack? This has been wreaking havok to the load balancers (both Rancher’s and otherwise).

Screenshot from inside a container:

Same behavior on (Cattle) 1.5.0 & 1.5.1-rc2, haven’t tried downgrading. I have restarted some of the hosts that host the non-responding to ping containers, no luck.

Solution, for any other unlucky soul:

  • Disconnect & Delete Host
  • Stop & Remove all rancher/* containers
  • Add Host

  • Simply stopping and restarting the host does not work (i.e. reboot or docker restart).
  • Simply removing the agent and adding the host does not work.

Inspiration credit: https://github.com/rancher/rancher/issues/8187

@localhost please ping/find me on https://slack.rancher.io if you happen to have this setup in the problematic state.

1 Like