Re: Problems with making hardlink-based backups
Thanks for your suggestion, and I have heard of rsnapshot.
Although, actually removing older snapshot directories isn't really the problem.
The problem is, if you have a large number of such backups (perhaps
one per server), then finding out where harddrive space is actually
being used, is problematic (when your backup server starts running low
on disk space).
du worked pretty well with rdiff-backup, but is very problematic with
a large number of hardlink-based snapshots, which each have a complete
"copy" of a massive filesystem (rather than just info on which files
changed).
I guess I could do something like removing the oldest snapshot
directories from *all* the backups, until there is enough free space.
But that's kind of wasteful. Like, if I have one server that didn't
change much over 2 years, then I can only keep eg the last 2-3 weeks
of backups, because there is another server that has a huge amount of
file changes in the same period. And not being able to use "du" is
kind of annoying (actually, "locate" is also having major problems, so
I disabled it on the backup server).
That's why I started working on a set of pruning/unpruning scripts,
which basically "move" redundant info (the vast majority) over into
compressed files (with ability to move out again later). Kind of like
moving the snapshot-based approach closer to how rdiff-backup works
(but, not chewing up huge amounts of ram and being hard to diagnose).
That way admins can in theory more easily check where space is being
used (but at the cost of not having quick access to earlier complete
server snapshots).
But I assume there must be better existing ways of handling this kind
of problem, since backups aren't exactly something new.
On Thu, Aug 13, 2009 at 5:48 PM, Andrew
Sackville-West<andrew@farwestbilliards.com> wrote:
> On Thu, Aug 13, 2009 at 09:20:17AM +0200, David wrote:
>> Hi list.
> [...]
>> 3) Existing tools for managing hardlink-based snapshot directories
>> etc.
>
> maybe rsnapshot is what you're after. It does hardlinked snapshots
> with automagical deletion of older backups and configurable frequency
> etc. I quite like it, though I'm not using it for high-volume stuff.
>
> Once little caveat that always seems to get me: the daily won't run
> until you've completed enough hourlies, the weekly won't run until
> you've completed a week's worth of dailies, etc. Very disconcerting
> the first few days of use.
>
> A
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
>
> iEYEARECAAYFAkqENcwACgkQaIeIEqwil4bCHQCeLWJ+9UcjtYqyolT6kiK7kDLy
> R20Aniawf/KsnU2uEG7D+35DjoksUJgS
> =qhWD
> -----END PGP SIGNATURE-----
>
>
Reply to: