Failed to create pod sandbox. When network driver is flannel

Rancher
1

Hello.

I installed rancher couple of day ago. So I am not experienced user. Canal is the only network driver which works for me. Flannel and weave, they don’t. I would like to ask about flannel first. I am going to use terraform to configure rancher. And for me terraform is the key tool here. When I create new Custom cluster some Deployments/DaemonSets/Pods refuse to work. This is the error:

Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "588db269d3f4c89ae17111843862f3914b9067cebfd236d1e1ccb8926e24d3cc" network for pod "coredns-5d4666f5f-t247d": networkPlugin cni failed to set up pod "coredns-5d4666f5f-t247d_kube-system" network: error getting ClusterInformation: connection is unauthorized: clusterinformations.crd.projectcalico.org "default" is forbidden: User "system:node" cannot get resource "clusterinformations" in API group "crd.projectcalico.org" at the cluster scope, failed to clean up sandbox container "588db269d3f4c89ae17111843862f3914b9067cebfd236d1e1ccb8926e24d3cc" network for pod "coredns-5d4666f5f-t247d": networkPlugin cni failed to teardown pod "coredns-5d4666f5f-t247d_kube-system" network: error getting ClusterInformation: connection is unauthorized: clusterinformations.crd.projectcalico.org "default" is forbidden: User "system:node" cannot get resource "clusterinformations" in API group "crd.projectcalico.org" at the cluster scope]

This is my cluster yaml:

answers: {}
docker_root_dir: /var/lib/docker
enable_cluster_alerting: false
enable_cluster_monitoring: false
enable_network_policy: false
local_cluster_auth_endpoint:
  enabled: true
name: 43fr
rancher_kubernetes_engine_config:
  addon_job_timeout: 30
  authentication:
    strategy: x509|webhook
  authorization: {}
  bastion_host:
    ssh_agent_auth: false
  cloud_provider: {}
  ignore_docker_version: true
  ingress:
    provider: nginx
  kubernetes_version: v1.17.2-rancher1-2
  monitoring:
    provider: metrics-server
  network:
    mtu: 0
    options:
      flannel_backend_port: 7890
      flannel_backend_type: udp
      flannel_iface: ens160
    plugin: flannel
  private_registries:
    - is_default: true
      password: ffffffff
      url: artifactory.local/docker-remote
      user: admin
  restore:
    restore: false
  services:
    etcd:
      backup_config:
        enabled: true
        interval_hours: 12
        retention: 6
        safe_timestamp: false
      creation: 12h
      extra_args:
        election-timeout: '5000'
        heartbeat-interval: '500'
      gid: 0
      retention: 72h
      snapshot: false
      uid: 0
    kube-api:
      always_pull_images: false
      pod_security_policy: false
      service_node_port_range: 30000-32767
    kube-controller: {}
    kubelet:
      fail_swap_on: false
      generate_serving_certificate: false
    kubeproxy: {}
    scheduler: {}
  ssh_agent_auth: false

I’ve tried. (Apparently problem somewhere else but I tied)

network:
    mtu: 0
    options:
      flannel_backend_port: '7890'
      flannel_backend_type: udp
      flannel_iface: ens160
    plugin: flannel

and

network:
    mtu: 0
    options:
      flannel_backend_type: vxlan
      flannel_iface: ens160
    plugin: flannel

Probably this error could be fixed by changing something in ClusterRole but as I mentioned above I create cluster by terraform, I want my process to be automated. If this problem can be solved on yaml level it would perfect, because this is the easiest way.

2

kubelet log:

E0327 13:27:10.403515   89284 pod_workers.go:191] Error syncing pod cc70f593-0aa4-49b5-b3bb-2df9f2cc6809 ("coredns-994656974-hzbhf_kube-system(cc70f593-0aa4-49b5-b3bb-2df9f2cc6809)"), skipping: failed to "KillPodSandbox" for "cc70f593-0aa4-49b5-b3bb-2df9f2cc6809" with KillPodSandboxError: "rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"coredns-994656974-hzbhf_kube-system\" network: error getting ClusterInformation: connection is unauthorized: clusterinformations.crd.projectcalico.org \"default\" is forbidden: User \"system:node\" cannot get resource \"clusterinformations\" in API group \"crd.projectcalico.org\" at the cluster scope"
I0327 13:27:22.349181   89284 kuberuntime_manager.go:442] No ready sandbox for pod "coredns-994656974-hzbhf_kube-system(cc70f593-0aa4-49b5-b3bb-2df9f2cc6809)" can be found. Need to start a new one
W0327 13:27:22.350983   89284 cni.go:331] CNI failed to retrieve network namespace path: cannot find network namespace for the terminated container "012dec86c4c0d13d2b57f26db6637f798018dca1ebf49389cf519f8f61561a6c"
E0327 13:27:22.402724   89284 cni.go:385] Error deleting kube-system_coredns-994656974-hzbhf/012dec86c4c0d13d2b57f26db6637f798018dca1ebf49389cf519f8f61561a6c from network calico/k8s-pod-network: error getting ClusterInformation: connection is unauthorized: clusterinformations.crd.projectcalico.org "default" is forbidden: User "system:node" cannot get resource "clusterinformations" in API group "crd.projectcalico.org" at the cluster scope
E0327 13:27:22.403376   89284 remote_runtime.go:128] StopPodSandbox "012dec86c4c0d13d2b57f26db6637f798018dca1ebf49389cf519f8f61561a6c" from runtime service failed: rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod "coredns-994656974-hzbhf_kube-system" network: error getting ClusterInformation: connection is unauthorized: clusterinformations.crd.projectcalico.org "default" is forbidden: User "system:node" cannot get resource "clusterinformations" in API group "crd.projectcalico.org" at the cluster scope
E0327 13:27:22.403449   89284 kuberuntime_manager.go:898] Failed to stop sandbox {"docker" "012dec86c4c0d13d2b57f26db6637f798018dca1ebf49389cf519f8f61561a6c"}
E0327 13:27:22.403512   89284 kuberuntime_manager.go:676] killPodWithSyncResult failed: failed to "KillPodSandbox" for "cc70f593-0aa4-49b5-b3bb-2df9f2cc6809" with KillPodSandboxError: "rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"coredns-994656974-hzbhf_kube-system\" network: error getting ClusterInformation: connection is unauthorized: clusterinformations.crd.projectcalico.org \"default\" is forbidden: User \"system:node\" cannot get resource \"clusterinformations\" in API group \"crd.projectcalico.org\" at the cluster scope"