On Jan 27, G. Kracht [Hebbizz Services BV] illuminated :
Hi,
I would suggest BackupPC
http://backuppc.sourceforge.net/faq/BackupPC.html. Works like a charm.
Very clever pooling scheme. As far as I know there arent any
limitations to the number of files to backup.
This method *will* fail. It's not appropriate for data that is going to
change during the backup run. It is appropriate for data that is not
going to change during the run - backing up user data when the users are
not around, for example. Mail data files are always changing.
Why? It will miss any files that have changed or disappeared since the
backuppc (rsync) process was started. This will be the case for any
backup system which relies purely on rsync, tar or anything that generates
a list of files and then copies them. I have watched it fail on
desktop systems while any mail program is running. OS X's Mail.app, for
example. (However, that's over an IMAP connection, so it's moot - I still have
another copy of my mail.) Extending this to server side, where data is
continually changing increases the inconsistencies in your backups.
Also, if you have a lot of users, unless you split the backups up, rsync
will take time (and possibly a lot of time) to create its list of files
to be backed up. IIRC, when I did an rsync of some of our customer's web
directories, for a total of ~600GB, split into 100 sections, the rsync list
compilation time for each section was on the order of 15 minutes. Trying
to do it in one, or even ten runs was painful enough to be considered
failure. (This was changing data as well.)
I would hate to think how much longer it would have taken to do the same
thing on MailDir.
(part of) The fix - wrap a snapshot manager in between the filesystem
and the backup process. This will ensure that the backups you take are
consistent.
The best long term fix, IMO. Throw you data on a SAN and use its
replication or snapshot functionality.
./jp