Longhorn in GKE: Node Affinity

Longhorn
1

Hi!
I have a question concerning running Longhorn in GKE.

I already know that I cannot use the Google “Container Optimized OS” but need to run Ubuntu for Longhorn.

In my cluster, I have both Ubuntu and Container Optimized OS nodes and I used affinities to control which pods are scheduled on which node.

Can I use Longhorn in such an environment? Can I set the node affinity somewhere? If it works, is it correct that only the Ubuntu Nodes will be able to mount volumes managed by Longhorn?

Thank you for your help!

2

You can taint nodes for different purposes like non-longhron and longhorn workloads.

  • Update the settings of taint tolerance in longhorn to tolerant the taint for longhorn workload
  • Then, the longhorn workloads will be rescheduled to the tainted longhorn nodes

ref: Longhorn | Documentation

1 Like
3

If it works, is it correct that only the Ubuntu Nodes will be able to mount volumes managed by Longhorn?

Yes

1 Like
4

@PhanLe0110, I am not quite sure about this one.

Taint and tolerations means that the node will only accept workload from certain pods.

Tolerations is on the pod and says that it will accept being put on a node but not being explicit.

For example lets say I have 5 nodes.
ServerA
ServerB
ServerC
ServerD
ServerE

If I have a taint on ServerA and ServerB with something like storage=true:NoExecute ; storage=true:Noschedule. So as we expect this will not allow any pods to go on without having the toleration of storage=true:NoExecute ; storage=true:Noschedule.

So on the deployment we put on a toleration of:

storage=true:NoExecute
storage=true:Noschedule

what this means is that when the pod is scheduled it WILL be accepted on ServerA and ServerB but it would will ALSO be accepted on ServerC,D,E if they happen to get scheduled on there.

To control where the pod gets ‘attracted’ to you have to use node affinity and this is something I can’t find on LongHorn

5

@weiyentan
I see your use case! Thank you for the clarification.

Yeah, you are right. In order to solve this problem, Longhorn needs to allow users to specify node selector or node affinity so that Longhorn components only get deployed onto a specific group of nodes. We opt to go with node selector because it is simple and get the job done.

Another use case for this feature is that users have a cluster of both Linux nodes and Windows nodes. Users want to install Longhorn only on Linux nodes because Longhorn doesn’t work on Windows nodes.

We are working on this feature. The issue is tracked at [FEATURE] Set tolerations and node selectors for dynamically provisioned Longhorn workloads · Issue #2199 · longhorn/longhorn · GitHub. If nothing changes, it will be scheduled to release in Longhorn v1.1.1

1 Like