Re: Homebuilt NAS Advice

[Kushal Kumaran <kushal@locationd.net>]

Leslie Rhorer <lesrhorer@att.net> writes:

> On 8/5/2020 1:51 PM, deloptes wrote:
>
>> Imagine you have classical backup: daily incrementals, full weekly and full
>> monthly.
>
> 	This is not required with DAR.  One full backup is all that is
> required.  Anything else is a waste of time and space.  One can employ
> either differential or decremental backups, keeping a full history of
> all activity on the system.
>
>  Imagine you have retention for the full weekly 3 (until end of
>> month) and for full monthly 12 (until end of year).
>> You have to maintain 15 full backups and the 6 daily incrementals. How much
>> space is it, that you need for your backup storage?
>> This is why the question what is your active size. No imagine I have 2TB of
>> data, even if I compress this data - lets say with avg. 60% ratio it is
>> 800GB per full backup. 15 copies + means 12TB+. Of course if you have
>> video/audio like mp3, it is already compressed and ratio for the backup
>> compression goes down and space needed up.
>> Now here comes the trick with deduplication. The backup system makes one
>> full backup (800GB) and then keeps track of the bits that changed (it is
>> not that simple, but for the example). Only they are being backuped. Some
>> systems provide ratio of 90%. So to keep your 15+ copies with deduplication
>> ratio of ~80% you need about 3TB.
>
> 	You are talking about differential archiving at the binary
> level. DAR can employ delta binaries for both incremental and
> decremental backups, which means only the changes to the binary data,
> if any, is stored in subsequent backups, not the entire file.  There
> is no need for multiple full backups.  A full backup of my main array
> to hard drive media will take a minimum of 15 days.  Transferring
> across the 10G link to my backup server would take a minimum of 2.8
> days.  Although full backups of smaller systems will normally take
> less time, the fact is full backups on a regular basis are just not
> necessary, and especially not when using DAR.
>

Deduplication does a little more than that.  Deduplicated backup systems
can identify duplicated content across different files in addition to
the differential backups you mention.  This can help when you might have
identical (or close to identical) content across multiple files.  Things
like virtual machine disk images deduplicate rather well.  Also helpful
when you might have multiple users storing identical/similar files.

Sophisticated deduplication systems can even deduplicate across multiple
machines being backed up.

-- 
regards,
kushal