Re: Homebuilt NAS Advice
On 8/6/2020 11:15 AM, deloptes wrote:
Leslie Rhorer wrote:
A few copies of what size? ??The backup server is an exact mirror of the
main server, plus several T of additional files I don't need on the main
server.
The question is how much back in time you can go. If you have just a
mirror - it is one copy, so you can go one step back only.
Which is why I have offline archival backups in addition to the mirror.
Since this is primarily a media server, more than 21T of the files
will never change. More than 1/2 of the files are at least 5 years old:
root@RAID-Server:/# find /RAID/ | wc -l
382489
root@RAID-Server:/# find /RAID/ -mtime +365 | wc -l
352044
root@RAID-Server:/# find /RAID/ -mtime +730 | wc -l
286695
root@RAID-Server:/# find /RAID/ -mtime +1825 | wc -l
244813
root@RAID-Server:/# find /RAID/ -mtime +3650 | wc -l
22954
Almost all the files older than 1 year are new, not modified.
Note also that while there are 382489 files on the array, about 3000
are considerably larger than 1G, representing more than 70% of the space
on the drive. Not one of these files are duplicates at any level. The
majority of the remaining files (275729) are less than 10K each. Even
if every one were a duplicate, that only represents less than 1.5G of
uncompressed storage. The vast majority of those never change, either.
In reality, the savings would be zilch.
I don't argue about your approach, it is your design, your servers, but in
your case I would use definitely deduplication.
And save what? About 0.002%. There are maybe 100 duplicate files
larger than 50K on my server, excluding hard links and temporary files.
The hard links are handled expeditiously by DAR. Temporary files don't
get backed up.
If you have a problem with data transfer - use fiber, by that volume of data
10G is a must.
I do. The servers have 10G optical links between them. A full backup
to the RAID 6 array takes several days. A full backup to single drives
takes 2 weeks, because single drives are limited to about 800Mbps, while
the array can gulp down nerly 4Gbps. Nightly backups (via rsync) take a
few minutes, at most.
Reply to: