Re: definiing deduplication (was: Re: deduplicating file systems: VDO with Debian?)
On Thu, 2022-11-10 at 15:32 +0100, Thomas Schmitt wrote:
> Hi,
>
> i wrote:
> > > the time window in which the backuped data
> > > can become inconsistent on the application level.
>
> hw wrote:
> > Or are you referring to the data being altered while a backup is in
> > progress?
>
> Yes.
Ah I was referring to snapshots of backups.
> [...]
>
> > Yes, I'm re-using the many small hard discs that have accumulated
> > over the
> > years.
>
> If it's only their size which disqualifies them for production
> purposes,
> then it's ok. But if they are nearing the end of their life time, then
> i would consider to decommission them.
They aren't really disqualified. They've been replaced mostly because
electricity is expensive and because it can be unwieldy having so many
disks in a machine. What the end of their lifetime is is unpredictable.
Why would I throw away perfectly good disks.
> > I wish we could still (relatively) easily make backups on tapes.
>
> My personal endeavor with backups on optical media began when a
> customer
> had a major data mishap and all backup tapes turned out to be
> unusable.
That can happen with any media.
> Frequent backups had been made and allegedly been checkread. But in
> the
> end it was big drama.
> I then proposed to use a storage where the boss of the department can
> make random tests with the applications which made and read the files.
> So i came to writing backup scripts which used mkisofs and cdrecord
> for CD-RW media.
When CDs came out, they were nice for a short time because you didn't
need to switch between so many diskettes. It didn't take long for CDs
to be small, and DVDs had the same problem. A single CD instead of 20
disks is an improvement, but 20 or 100 CDs or DVDs is not, and it became
obvious that optical media is even more unwieldy than magnetic media.
At least you could just push a diskette into the drive and start reading
and writing; with CDs/DVDs, writing is not so easy.
> > Just change
> > the tape every day and you can have a reasonable number of full
> > backups.
>
> If you have thousandfold the size of Blu-rays worth of backup, then
> probably a tape library would be needed. (I find LTO tapes with up to
> 12 TB in the web, which is equivalent to 480 BD-R.)
Yes ... The problem is that for unknown reasons, the development seems
to have stopped at some point other than maybe for datacenters that
might have rooms full of tapes and robots to fetch them. Perhaps
physical limits made tapes with more capacity not really feasible.
>
> > A full new backup takes ages
>
> It would help if you could divide your backups into small agile parts
> and
> larger parts which don't change often.
> The agile ones need frequent backup, whereas the lazy ones would not
> suffer
> so much damage if the newset available backup is a few days old.
Nah, that would make things very complicated. Rsync figures it
automatically.
> > I need to stop modifying stuff and not start all over again
>
> The backup part of a computer system should be its most solid and
> artless
> part. No shortcuts, no fancy novelties, no cumbersome user procedures.
That's easier said than accomplished ...
Reply to: