Where my storage goes in Longhorn?

Longhorn
1

I spin up 3 nodes just for storage. The nodes are VM’s base on Unbuntu 20.10. Every node has 150Gig of disk storage. All of the 3 nodes are now ready an available in Longhorn but at the Dashboard i only see 128gig scheduled. Where the other storage goes? I thought that if you spin up 3 machines with each 150 GB of storage you would at least have 400 GB of storage available in Longhorn.

I am new to Rancher, K3s and Longhorn so maybe i am doing something wrong here.
I appreciate your help in this.

2

Longhorn uses space allocated at /var/lib/longhorn. The directory includes engines, configurations, and replicas. Depending on number of workload/replica, data will be sync accordingly in the subdirectories.

3

Thanks for answering but it does not answering my question where are the storage goes that i attached to the 3 storage VM’s that i created. Let me put it like this:
In longhorn i dedicated 450 GB to storage and the result is only 128GB available for my projects. That doesn’t give me the satisfaction i was hoping and looking for. And to answer your question further, there is nothing right now running. I just set up the cluster and i’m experimenting. So the cluster is empty.

4

When deployed 3 nodes with 150GB each, not all the space are dedicated to Longhorn. Longhorn reserved some spaces for systems and other applications. In your case, 128GB is available in each node. When creating volume of n replica, Longhorn will replicate the volume to the different nodes for high availability. Creating a 10GB volume with replica of 3, It will take 10GB of data in each node if disk is fully written. Additional space will take up when you do snapshot/backup. You can find more information on Node Space Usage in the documentation below:

5

When i go to the Longhorn dashboard. The circle in the middle shows that there is 125GB. To me that does not mean that i have 125 on each node. Else it would say that i have a total of 375 GB schedulable. It says also that besides the 125GB, 65 GB is reserved and 28,4 is being used. I compare Longhorn with a raid 5 configuration. If this was a raid 5 configuration i would end up with 300 GB of total storage. Now i don’t have even half of it. That’s too much space that i lost to Longhorn so i won’t recommend it

6

I found the cause of the problem.
Somehow Unbuntu 20.10 doesn’t use the total size of the volume i created. Unbuntu only use 73 gig.
I extend the volume and now Longhorn is showing that i can use a total amount of 345 gig. That is what i expected and now its OK.

2 Likes
7

Hello @tando, could you explain how do you extend the volume of each node? Thanks!

8

@Azulado

df -h

that shows usage being around 75gb

Run the following:

sudo vgdisplay
sudo lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv
sudo resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv

then again with

df -h

You will see the partition has been expanded to 100% of the free space
Give longhorn a minute or two, and it’ll update the dashboard :slight_smile: