Re: Storage server
On Tue, 11 Sep 2012 16:29:22 +0100
Jon Dowland <jmtd@debian.org> wrote:
> Denis' answer is very good, I won't re-iterate his points.
Thanks. And also thanks for pointing out the Hardlinks thing, I
over-read the "lots of small files" part in Velkjos Mail.
Anyway, I have some comparison data. I have a backup server that saves
data from 5 other server at our hosting company using rsnapshot. The
backups are kept for 14 days.
rsnapshot:
The Backup has 186GB. 51 GB for the full backup (daily.0) and
about 11GB for each incremental backup (daily.1 - daily.13). The backup
includes typical small webserver files but also big logfiles and two
ZOPE Databases (ZEOs) with about 5GB each.
bup:
I imported them with "bup import-rsnapshot", the overall size is 15GB
(for all 14 days) which is quite amazing. Anyway the lack of a
possibility to delete old backup versions is (for me) a major drawback.
What I liked was the possibility to mount the Backup with FUSE. After
mounting the Backup one can access every backup generation as normal
files. Each generation has its own folder with a timestamp. The files
inside the backup have no metadata (timestamp is always 1.1.1970, etc.).
The only way I can think of at the moment to get rid of old backup
generations using bup is to mount (FUSE) the backup restore all backup
generations you want to keep to an additional drive, delete the bup git
repository, create a new one and backup the restore again. Of course
this might take a lot of additional space on you disk for the
(temporary) restore which might not be available. If any has some
better approach I would love to hear.
obnam:
With obnam I made a backup of daily.0 (51GB). There was nearly no
reduction in the size for the first backup run (47GB). The next backup
run (one day later, which creates 11GB new data with rsnapshot) has only
added a few MB and therefore was pretty fast.
The repository approach of obnam comes very handy. You can pull or push
backups to the repository server and can access the backups from any
other machine (if you have SSH access). Configuration is not necessary
but a small config containing some default parameters comes in handy:
[config]
repository = sftp://192.168.1.10/backup/obnam/
log = /var/log/obnam.log
log-level = warning
client-name = dx
If you now run "obnam backup /var/www" the backup of /var/www will be
pushed to the repository. obnam locks the repository for the client so
one cannot accidentally run two backups of the same host (client) at
the same time. Running several backups of different hosts is no problem.
During a backup run obnam makes "snapshots" every few 100MB so if the
backup fails (e.g. disconnect from the repository server) the backup
can be resumed from the last snapshot.
A nice feature is some kind of built in nagios plugin:
obnam nagios-last-backup-age --warn-age=1 --client=dx
OK: backup is recent. last backup was 2012-09-12 10:05:47.
obnam nagios-last-backup-age --warn-age=1 --client=backup
WARNING: backup is old. last backup was 2012-09-11 18:03:43.
obnam nagios-last-backup-age --client=cat --critical-age=1
CRITICAL: backup is old. last backup was 2012-09-11 17:01:23.
The restore is a bit more complex, as there is (at the moment) no FUSE
filesystem available for obnam. Instead you need to know the name of
the file/folder and in which backup generation your file/folder exists.
"obnam generations" shows all available backups:
101 2012-09-11 18:01:07 .. 2012-09-11 18:02:10 (26474 files,
8598496965 bytes)
108 2012-09-12 10:05:47 .. 2012-09-12 10:06:36 (26474 files,
8598500897 bytes)
Then you can use "obnam ls --generation=101" to show the files.
rdiff-backup:
If have no real comparison data for rdiff-backup but I expect similar
results as with obnam (about 50GB for the first backup, only several MB
for each following daily backup).
rdiff-backup can (like bup) mount the backup (all generations) using
FUSE.
Best regards
Denis Witt
Reply to: