In one of the longhorn presentations recently given one use case was the ability to have the block distributed between AZ’s e.g. in aws.
Ive been reading the documents but cannot find any details on this or how to ensure a block replica is in another zone.
Is it by default relying on “topology.kubernetes.io/zone” labels on a node? Or how can we validate it is infact keeping replicas across zones?
Thanks in advance
Yes, you are correct.
For k8s version < 1.17, set label like this failure-domain.beta.kubernetes.io/zone=us-east-1c
For k8s version >=1.17, set label like this topology.kubernetes.io/zone=us-east-1c
k8s official doc is at k8s zone label.
If Replica Zone Level Soft Anti-Affinity is set to false, this means the new replica will be scheduled on node with different zones. To validate this, you can check all the replicas should on nodes with different zones.
Longhorn schedule policy doc has more details about it. Please take a look.