Kube-system FailedCreatePodSandBox

Hello I am very new to rancher and kubernetes.
I have a rancher server 2.4.4 and added a cluster node (running RedHat Atomic OS) using below command
sudo docker run -d --privileged --restart=unless-stopped --net=host -v /etc/kubernetes:/etc/kubernetes -v /var/run:/var/run rancher/rancher-agent:v2.4.4 --server https:// --token --ca-checksum --etcd --controlplane --worker.

After that when i tried to deploy a simple hello-world app and getting bellow error
“kube-system Warning FailedCreatePodSandBox coredns-7c5566588d-vzgqx (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container “b6f5d18da314215755b5d5111ec58a277ad244469665cc21d785d37ac953084d” network for pod “coredns-7c5566588d-vzgqx”: networkPlugin cni failed to set up pod “coredns-7c5566588d-vzgqx_kube-system” network: stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/”

I have checked the cluster node and didnot see any flannel interface created.
All Pods on the system

kubectl get pods --all-namespaces
cattle-system cattle-cluster-agent-5f5bd95c56-qm4w5 0/1 ContainerCreating 0 178m
cattle-system cattle-cluster-agent-6cc55c58c5-5xlnt 0/1 ContainerCreating 0 133m
cattle-system cattle-node-agent-rv75z 1/1 Running 0 105m
cattle-system kube-api-auth-7s8s5 1/1 Running 0 105m
ingress-nginx default-http-backend-5b7bbcd7c9-82kw2 0/1 ContainerCreating 0 130m
ingress-nginx default-http-backend-67cf578fc4-h5gf7 0/1 ContainerCreating 0 178m
ingress-nginx nginx-ingress-controller-997sh 1/1 Running 0 101m
kube-system canal-zsz25 0/2 Init:CrashLoopBackOff 24 102m
kube-system coredns-7c5566588d-vzgqx 0/1 ContainerCreating 0 102m
kube-system coredns-autoscaler-65bfc8d47d-wqjc5 0/1 ContainerCreating 0 178m
kube-system coredns-autoscaler-7dcdbdbdcc-ksnsb 0/1 ContainerCreating 0 130m
kube-system metrics-server-6b55c64f86-6x6xs 0/1 ContainerCreating 0 178m
kube-system metrics-server-7d6b88fb7c-z88lp 0/1 ContainerCreating 0 130m
kube-system rke-coredns-addon-deploy-job-jbrjj 0/1 Completed 0 102m
kube-system rke-ingress-controller-deploy-job-kqbkx 0/1 Completed 0 101m
kube-system rke-metrics-addon-deploy-job-t2j5k 0/1 Completed 0 102m
kube-system rke-network-plugin-deploy-job-v5zrh 0/1 Completed 0 103m

I am not sure how to move forward.

I have found the issue. /usr is a readonly filesystem on RedHat Atomic Host.

+++++++++++++++++++++++ Error +++++++++++++++++++++++++
skipping: failed to “StartContainer” for “flexvol-driver” with RunContainerError: “failed to start container “90351ed1b162f14d70d160a39d349d5786a228eafd4a102609ec772e6b11b846”:
Error response from daemon: error while creating mount source path ‘/usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds’: mkdir /usr/libexec/kubernetes: read-only file system”

Changed the flexvol-driver-path in cluster config as outlined in https://github.com/rancher/rke/issues/1744.