Re: dear lazyweb: looking for a special backup tool
On Tuesday 17 November 2015 03:14:50 Thomas Schmitt wrote:
> Hi,
>
> Christoph Anton Mitterer wrote:
> > I'm looking for a backup solution with quite some specific needs,...
>
> My scdbackup with xorriso as backend might nearly do what
> you want:
> http://scdbackup.sourceforge.net/main_eng.html
>
> It initially demands some configuration effort
> http://scdbackup.sourceforge.net/examples.html
> http://scdbackup.sourceforge.net/README
>
> xorriso is available as Debian package, or as source tarball
> https://www.gnu.org/software/xorriso/xorriso-1.4.0.tar.gz
>
> > - hard links must be retained
> > - it must be possible to backup to split media (e.g. mutliple CDs)
>
> This is not easy to achieve.
> Consider one hardlink sibling in one fileystem and the other
> in the next filesystem.
>
> > - file times, owners (ideally as IDs and names), permissions,
> > XATTRS, ACLs must all be retained
>
> scdbackup records them in scripts with setfattr and setfacl,
> which it stores in data files in the backup.
> xorriso records them too, but restoring those will need xorriso,
> because Linux does not interpret AAIP (which is not astounding
> because it is a libisofs extension to ISO 9660).
>
> ISO 9660 has the advantage of being readable nearly everywhere
> as long as you restrict yourself to single-session and data
> files smaller than 4 GiB. (BSD and Solaris ended development of
> their ISO 9660 readers long ago. Meanwhile one can smell their
> fermentation state.)
>
> > - I want always *full* files to be backuped
> > - a single file shouldn't be split over multiple backup media
> > (unless this isn't possible otherweise, because all targets are
> > smaller than the file size)
>
> Putting together the pieces of large files is of course a pain
> at restore time.
>
> > - ideally the program would offer two modes:
> > - either trying to keep "neighbouring" files (i.e. those that are
> > close to each other in the directory hierarchy) closely on the
> > split target mediums
>
> You mean those with neighboring names, i assume.
> (More or less the same as alphabetic ordering by name.)
>
> > - or trying to be as space efficient as possible (i.e. place files
> > so that space is used most efficiently
>
> After a few years of cramming files, i switched to alphabetic.
>
> > - catalogues should be made, of both, all files and the files on a
> > certain medium, also as a help in the disaster case
>
> Not to forget checksums.
> In most dangerous storage environments one should consider to
> make several identical copies of the media and have means to
> recognize good blocks.
>
> > - as I've said, incremental backups should be possible,... but that
> > should also work when I move files around
>
> So timestamps are not enough.
> scdbackup has two ways of incremental backups:
> - If inode numbers are persistent and ( devide numbers are persistent
> or mount points always lead to the same filesystems), then it is
> possible to decide by the outcome of stat(2).
> - Much slower is decision on base of MD5 of file content.
>
> > Well I guess it wouldn't be too difficult to script most of this,
>
> scdbackup needed about seven years of try-and-error.
> It can be done faster if you have a clear plan.
> Beware of being sucked into neighboring topics like DVD burning
> or backup-grade ISO 9660 production.
>
>
> Have a nice day :)
>
> Thomas
I have been using amanda (Advanced Maryland Automatic Network Disk
Archiver) since 1998 or 99. Setup here to use files on a BIG HD that
are its "tape library", I have found such a setup to be both much faster
when it comes to recovery, AND considerably more dependable than any
tape format I can afford. Amanda has a different basic format, mixing
backup levels in an attempt to use about the same amount of media per
run. A typical run over the 4 machines live on my home network will run
to about 40Gb, which it does in about 3.5 hours in the middle of the
night. I do a 4 day cycle so I get a level0 full backup of every
disklist entry at some point in those 4 days, mixed with incremental,
only whats changed, in between times. And I can reach back as far as 30
days before that part of the disk is reused. So if I find something
valuable has been erased, I have 30 days to discover it and recover it.
There are tools which can make offsite, forever storage for amanda users.
But I'm not recording business transactions I might have to recover for
the Infernal Revenoo Service, I'm just a retired hobbiest making a stick
or two of furniture when the mood strikes me.
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>
Reply to: