This happened again and no way to bring it back. The only option is reinstall Rancher server and register the existing k8s to it. The downside is the k8s can’t be upgraded.
This is annoying and seems no update to similar cases in the forum.
2023/03/20 02:26:23 [INFO] Stopping cluster agent for c-4jnqb 2023/03/20 02:26:23 [ERROR] failed to start cluster controllers c-4jnqb: context canceled