Rke2 with ipv6 not working

I am setting up a RKE Cluster with rke2 binary and using the below config for worker and master. Everything goes fine, but at the end I am seeing that ‘calico-node-5xjnx’ is in crashloop with this error:

Defaulted container "calico-node" out of: calico-node, flexvol-driver (init), install-cni (init)
W1108 17:29:21.843958       9 feature_gate.go:241] Setting GA feature gate ServiceInternalTrafficPolicy=true. It will be removed in a future release.
2023-11-08 17:29:21.847 [INFO][9] startup/startup.go 432: Early log level set to info
2023-11-08 17:29:21.847 [INFO][9] startup/utils.go 126: Using NODENAME environment for node name k8s-slave-03
2023-11-08 17:29:21.848 [INFO][9] startup/utils.go 138: Determined node name: k8s-slave-03
2023-11-08 17:29:21.848 [INFO][9] startup/startup.go 94: Starting node k8s-slave-03 with version v3.26.1
2023-11-08 17:29:21.848 [INFO][9] startup/startup.go 437: Checking datastore connection
2023-11-08 17:29:21.855 [INFO][9] startup/startup.go 461: Datastore connection verified
2023-11-08 17:29:21.855 [INFO][9] startup/startup.go 104: Datastore is ready
2023-11-08 17:29:21.860 [INFO][9] startup/customresource.go 102: Error getting resource Key=GlobalFelixConfig(name=CalicoVersion) Name="calicoversion" Resource="GlobalFelixConfigs" error=the server could not find the requested resource (get GlobalFelixConfigs.crd.projectcalico.org calicoversion)
2023-11-08 17:29:21.870 [INFO][9] startup/startup.go 490: Initialize BGP data
2023-11-08 17:29:21.870 [INFO][9] startup/autodetection_methods.go 103: Using autodetected IPv4 address on interface ens160: 10.2.144.41/20
2023-11-08 17:29:21.871 [WARNING][9] startup/autodetection_methods.go 99: Unable to auto-detect an IPv6 address: no valid IPv6 addresses found on the host interfaces
2023-11-08 17:29:21.871 [WARNING][9] startup/startup.go 545: Couldn't autodetect an IPv6 address. If auto-detecting, choose a different autodetection method. Otherwise provide an explicit address.
2023-11-08 17:29:21.871 [INFO][9] startup/startup.go 396: Clearing out-of-date IPv4 address from this node IP="10.2.144.41/20"
2023-11-08 17:29:21.871 [INFO][9] startup/startup.go 400: Clearing out-of-date IPv6 address from this node IP=""
2023-11-08 17:29:21.876 [WARNING][9] startup/utils.go 48: Terminating
Calico node failed to start
cluster-cidr:
  - 192.168.0.0/16
  - 2001:cafe:168:0::/56
service-cidr:
  - 192.169.0.0/16
  - 2001:cafe:169:1::/112
cluster-dns: 192.169.0.10
write-kubeconfig-mode: "0644"
node-ip:
  - <node ipv4>
  - <node ipv6>
node-external-ip:
  - <node ipv4>
  - <node ipv6>
cni: calico
server: <master-node-ip>:9345
token: <master node token>
node-label:
  - nodetype=worker
kubelet-arg:
  - --kube-reserved=cpu=1000m,memory=1000Mi,ephemeral-storage=5Gi
  - --system-reserved=cpu=1000m,memory=1000Mi,ephemeral-storage=5Gi
  - --eviction-hard=imagefs.available<10%,imagefs.inodesFree<10%,memory.available<1Gi,nodefs.available<10%,nodefs.inodesFree<10%,pid.available<10%
  - --max-pods=100