Fail to get target node ID

Longhorn
1

Hi

I am installing Longhorn using Helm on a K3s cluster. All pods are running however I am unable to use LongHorn due to the following message:

fail to get the parameters: fail to get target node ID: cannot find a node that is ready and has the default engine image longhornio/longhorn-engine:v1.3.1 deployed

I am not sure where to look to fix,

Any suggestions please.

Thank you

JT

2

Based on the messages returned, it appears that Longhorn could not find any nodes that had successfully deployed the engine image.

Could you use below three commands to check manager pod log, nodes status and engineimage deployment status.

  • Manager pod log: support bundle files (How to generate a Longhorn Support Bundle)
    • Or kubectl logs <longhorn-manager-pod-name> -n longhorn-system
  • Node stauts: kubectl get nodes.longhorn.io -n longhorn-system
  • Engineimage deployment status: kubectl describe engineimages.longhorn.io -n longhorn-system
3

Hi

I encounter the same issue. I am deploying longhorn with version 1.5.0. There are total 8 worker nodes in my k8s cluster, 3 nodes are tainted and labeled for deploying longhorn, the other 5 nodes are used for deploying normal applications.

longhorn-node-01
longhorn-node-02
longhorn-node-03

normal-node-01
normal-node-02
normal-node-03
normal-node-04
normal-node-05

After custom configuration editing, I got the current pods list as below. Maybe you should notice the number of pod instance for every deployment.

NAME                                                READY   STATUS    RESTARTS      AGE
csi-attacher-556df7d7f5-j6jct                       1/1     Running   0             83m
csi-attacher-556df7d7f5-q6rnn                       1/1     Running   0             83m
csi-attacher-556df7d7f5-tfwmn                       1/1     Running   0             83m
csi-provisioner-857f564c64-nqvkd                    1/1     Running   0             83m
csi-provisioner-857f564c64-rqndw                    1/1     Running   0  83m
csi-provisioner-857f564c64-w6c99                    1/1     Running   0  83m
csi-resizer-74899dc9d4-67fcc                        1/1     Running   0             83m
csi-resizer-74899dc9d4-bhdnj                        1/1     Running   0             83m
csi-resizer-74899dc9d4-gk2qn                        1/1     Running   0             83m
csi-snapshotter-d6cb885f8-c8dmp                     1/1     Running   0             83m
csi-snapshotter-d6cb885f8-klcv7                     1/1     Running   0             83m
csi-snapshotter-d6cb885f8-zptd5                     1/1     Running   0             83m
engine-image-ei-d911131c-fph6m                      1/1     Running   0             83m
engine-image-ei-d911131c-fx2h7                      1/1     Running   0             83m
engine-image-ei-d911131c-gtvz5                      1/1     Running   0             83m
engine-image-ei-d911131c-vdwm6                      1/1     Running   0             83m
engine-image-ei-d911131c-zlwft                      1/1     Running   0             83m
instance-manager-3ccc00a7297a55afd21e8f69fa3c05d7   1/1     Running   0             96m
instance-manager-866324202974948c2874603fee5590c0   1/1     Running   0             96m
instance-manager-8bf2fd2b1acd966246f159477445164c   1/1     Running   0             96m
longhorn-csi-plugin-4krlw                           3/3     Running   0             83m
longhorn-csi-plugin-bjk65                           3/3     Running   0             83m
longhorn-csi-plugin-frbfv                           3/3     Running   0             83m
longhorn-csi-plugin-plpd9                           3/3     Running   0             82m
longhorn-csi-plugin-vtqht                           3/3     Running   0             82m
longhorn-driver-deployer-54bd6969b4-qrtxh           1/1     Running   0             83m
longhorn-manager-hg9xw                              1/1     Running   0             83m
longhorn-manager-l9spk                              1/1     Running   0             83m
longhorn-manager-rcdx8                              1/1     Running   0             83m
longhorn-ui-7c6577c4c6-c8nqb                        1/1     Running   0             83m
longhorn-ui-7c6577c4c6-sq7t4                        1/1     Running   0             83m

As it shows I got 5 engine-image-ei pods. But these pods are deployed on normal nodes not the
3 tainted and labeled nodes. I wonder is it deployed right.

command outputs:

# kubectl get nodes.longhorn.io -n longhorn-system
NAME                               READY   ALLOWSCHEDULING   SCHEDULABLE   AGE
longhorn-node-01   True    true              True          116m
longhorn-node-02   True    true              True          116m
longhorn-node-03   True    true              True          116m

# kubectl describe engineimages.longhorn.io -n longhorn-system
...noise...

  Node Deployment Map:
    longhorn-node-01:  false
    longhorn-node-02:  false
    longhorn-node-03:  false
    normal-node-01:  true
    normal-node-02:  true
    normal-node-03:  true
    normal-node-04:  true
    normal-node-05:  true

...noise...

Besides, there are many debug logs in longhor-manager that lonhorn-nodes don’t have the engine image longhornio/longhorn-engine:v1.5.0

4

Maybe I misunderstand achitecture of longhorn. I remove taints for longhorn nodes and just use node selector to deploy longhorn-manager and other related pods.

After installing longhorn. I got longhorn-manager and other pods running on selected node. But all nodes in my cluster have longhorn-csi-plugin and engine-image-ei daemonset running. I wonder is this result expected as designed.