DAS Storage

Archive SUSE Enterprise Storage
1

Hello

For a backup/archiving cluster would using a sufficiently powerful server with Direct Attached Storage (Thinking Dell MD1400 over 12Gb SAS) for the OSD’s be sufficient?

Thanks
G

2

We are using MD1400s in our SES deployment. The thing about SES/Ceph is that “sufficiently powerful server” is very workload specific. So my general advice is have a very good idea of your workload in terms of I/O inside the servers and over the network.

Andrew

3

Thanks

Are you happy with the performance on the MD’s with your workload? We are looking at using it for archiving based on file age, so the first few archiving runs will be write intensive and it will slow down over time.

4

As for me, I don’t like the idea of using DAS, and would prefer to invest the same money into each SES cluster node itself (CPUs, RAM, HBAs, HDDs and 60-90 HDD SAS case with expanders).
My thoughts regarding ‘cons’ are:

  • You will require 3-4 pairs of “server+DAS” just to start. Such combination usually costs more, than 3-4 servers with HDDs ‘integrated’.
  • The DAS controller and additional SAS hop (internal or external) might add something to latency (which is less important for backup) and be a throughput bottleneck (which is much important). Direct HDD attach to system HBA usually is faster in both terms (throughput and latency).
  • In the case of software OSD failure / DAS crash You will have to resync whole node data, which will take longer time. The data being spread over multiple OSDs on node might partially survive, so, the resync time will be less. That means the ‘many-OSD node’ will be less vulnerable in some cases.

Thoughts regarding ‘pros’:

  • Single OSD per node is MUCH easier to monitor and troubleshoot.
  • Single HDD failure will not propagate to OSD level, as the DAS will manage that event by itself (using RAID).
5

We have run some initial benchmarks on SES3 and they were good but we are using NVMe drives as a cache tier (outside of SES) so they’re not typical of only having the MD1400s.
Deepsea has some issues with NVMes that we are getting worked out on SES4 so I haven’t run any more benchmarks yet.

Andrew

6

MD1400s have next to no brains in them and Ceph/SES doesn’t want RAIDed disks anyway. The topology for our systems are onboard disks go through one SAS controller and the MD1400s go through another that’s actually a bit more powerful.

If you have MD1400s then you probably have Dell servers to go with them rather than some SuperMicro special. They are a cheap way to add 12 disks to a host with reasonable performance.

Andrew

7

Thanks for this info

G

8

Thanks polezhaevdmi, interesting information.

Yes Andrew, we do Dell servers, due to some procurement regulations it is much quicker to get them. I was thinking to use two SAS controllers with half of the disks in the MD1400 on each controller and presenting the disks as an OSD per disk.

I have some spare servers with MD1400’s in store and will do some testing if I get the time.

G

9

Morning!

aferris, You are right. Sorry, I read the MD1400 specification inaccurately and made a wrong conclusion it has own RAID controller.
Still, I don’t like the MD1400, as it has only 12x 3.5 HDD bays, and the backup application usually means the demand of A LOT of space. As for pure Dell, the 4 node SES cluster with “Dell PE R530 + MD3060 + 10TB NLSAS” tandems looking much interesting from my point of view.

Cheers,

10

Morning.

I did not know about the MD3060, this looks much more viable.

Thank you
G