Hi,
I need to make backups of my email archive on SuSe11. The archive consists of around 20 million files and is currently about 2 TB in size.
Is it a viable proposition to back this stuff up to a portable USB 3.0 drive. I assume there are drivers available in SuSE but I don’t know
what the performance would be like. I have tried this on a fairly fast tape drive but it takes days to complete.
If this is not an option, what would you do?
Regards
John 
Personal opinion:
I have terrible, terrible luck with many little files when writing.
Reading them doesn’t seem to be terrible, but writing millions of small
files takes, seemingly, forever. Overhead, fine, whatever. As a result I
would probably try to use tar to grab the files, then have it write one
nice stream of data to the USB drive (or wherever).
tar -cvf /path/to/backup-location/gw-`date +%s`.tar /path/to/gw/filesIt may be worthwhile to add some compression in there too, but that will
slow things down, so I’d probably start with merely creating the tar.
Note: huge files scare me more than small ones, so maybe worthwhile to
create a tar per some smaller subset of GW data, so corruption of one
block of data on the USB drive (because those are so well-known for
resiliency…) doesn’t lose your entire backup.
Another silly thought: If this is meant as a backup to be used at any
point in time to recover things, how about using rsync to copy data there?
You can trivially set it up to copy whatever isn’t already there so that
your USB copy is kept up to date pretty quickly with the live version.
The result, after the initial 2 TB sync, should be pretty quick
synchronization to remove old and add new stuff. Just be SURE you do the
rsync the right way, particularly if you use the remove-stuff option.
Tutorials on rsync are prevalent online.
–
Good luck.
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…
Cant’ say much. But I’ve had better luck using Gigabit Ethernet.
We use an internal hot swap bay plugged to a standard sata port in the
server. You simply need to have a drive in the bay when booting and
then mount it manually when you need it. Dismount it and change out to
a new drive when you want. Just make sure to always physically have a
drive in the bay. We spent a lot of time trying to get usb 3 to work
and never had any luck.
spartanvb’s Profile: https://forums.novell.com/member.php?userid=57588
View this thread: https://forums.novell.com/showthread.php?t=479993