Nutanix CSI Volume Driver

Rancher 2.x
1

Hi,

currently I am trying to install the > Nutanix CSI Volume Driver 1.1 on a Kubernetes cluster, which was provisionied by RKE. The underlying GuestOS is RancherOS.

On my woker nodes I installed the iSCSI package with the following commands:
ros service enable open-iscsi
ros service up open-iscsi

Also I have added /lib/modules to the extra_binds of the kubelet and changed the volume paths of the ntnx-csi-node.yaml to /var/lib/kubelet/volumeplugins, /opt/rke/var/lib/kubelet/plugins/com.nutanix.csi/ and /opt/rke/var/lib/kubelet/

I can’t find any documentation about kubelet and two paths /var/lib/kubelet and /opt/rke/var/lib/kubelet. Maybe someone can explain the concept behind it?

The CSI provisioner and also the CSI node plugin pods are running. And I can create a PVC and the PV is created as expected, but it can’t be attached to the pod. I get the following error:

MountVolume.MountDevice failed for volume “pvc-1f686380-0639-4e85-92cf-462fdda7f5f4” : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name com.nutanix.csi not found in the list of registered CSI drivers

Is there any documentation how I can add a custom provisioner to Rancher?

Best regards,
sike

2

In the meantime I solved this problem. Therefore I changed the volumes to the following:

  • name: registration-dir
    hostPath:
    path: /opt/rke/var/lib/kubelet/plugins_registry/
    type: Directory
  • name: plugin-dir
    hostPath:
    path: /opt/rke/var/lib/kubelet/plugins/com.nutanix.csi/
    type: DirectoryOrCreate
  • name: pods-mount-dir
    hostPath:
    path: /opt/rke/var/lib/kubelet/
    type: Directory

Additionally I had to change the DRIVER_REG_SOCK_PATH environment variable of the driver-registrar container to /opt/rke/var/lib/kubelet/plugins/com.nutanix.csi/csi.sock.

But now I have a fundamental problem. The open-iscsi service is also bundled as container and therefore the iscsiadm script looks like the following:
#!/bin/sh

exec system-docker run --rm --privileged
–pid host
–net host
–ipc host
-v /mnt:/mnt:shared
-v /media:/media:shared
-v /dev:/host/dev
-v /run:/run
-v /etc/iscsi/nodes/:/etc/iscsi/nodes/
-v /etc/iscsi/send_targets/:/etc/iscsi/send_targets/
iscsi-tools $(basename 0) @

So the csi-node-ntnx-plugin container can not use the script unless I mount the system-docker and the socket file. Is there any solution for this problem? Does nobody used the open-iscsi service before? What is the use case for this service?