I am not able to add new nodes in my user RKE cluster which is managed by rancher cluster. I initially created user cluster using terraform rke provider. But due to some issues with SSL certs, rancher cluster became unusable. So, I created a new rancher cluster and imported user cluster to this new Rancher cluster. Now I am not able to add any new node in user cluster. I even tried to run same rancher-agent docker command that was being run for other nodes as part of the user-agent.
Any idea or insight on how can I add new nodes in user cluster?
After I run agent docker command, I see that node is stuck in this state - “Waiting to register with Kubernetes”. On the agent side, I just see these logs -
INFO: Arguments: --server https://xx.xx.xx --token REDACTED --worker --internal-address 10.0.103.203
INFO: Environment: CATTLE_ADDRESS=10.0.103.203 CATTLE_INTERNAL_ADDRESS=10.0.103.203 CATTLE_NODE_NAME=ip-10-0-103-203 CATTLE_ROLE=,worker CATTLE_SERVER=https://xx.xx.xx CATTLE_TOKEN=REDACTED
INFO: Using resolv.conf: nameserver 127.0.0.53 options edns0 search us-west-2.compute.internal
WARN: Loopback address found in /etc/resolv.conf, please refer to the documentation how to configure your cluster to resolve DNS properly
INFO: https://xx.xx.xx /ping is accessible
time="2021-04-21T21:29:20Z" level=info msg="Rancher agent version v2.4.4 is starting"
time="2021-04-21T21:29:20Z" level=info msg="Option customConfig=map[address:10.0.103.203 internalAddress:10.0.103.203 label:map[] roles:[worker] taints:[]]"
time="2021-04-21T21:29:20Z" level=info msg="Option etcd=false"
time="2021-04-21T21:29:20Z" level=info msg="Option controlPlane=false"
time="2021-04-21T21:29:20Z" level=info msg="Option worker=true"
time="2021-04-21T21:29:20Z" level=info msg="Option requestedHostname=ip-10-0-103-203"
time="2021-04-21T21:29:20Z" level=info msg="Listening on /tmp/log.sock"
time="2021-04-21T21:29:20Z" level=info msg="Connecting to wss://xx.xx.xx /v3/connect/register with token zdkz24s74ws6b8zrx5lrtptr4tn49s6gd98w7zbxj8l8lh9bn5gxx5"
time="2021-04-21T21:29:20Z" level=info msg="Connecting to proxy" url="wss://xx.xx.xx /v3/connect/register"
time="2021-04-21T21:29:20Z" level=info msg="Starting plan monitor, checking every 120 seconds"