Re: Storage server
On Wed, Sep 12, 2012 at 10:46:37AM +0200, Denis Witt wrote:
> On Tue, 11 Sep 2012 16:29:22 +0100
> Jon Dowland <jmtd@debian.org> wrote:
>
> > Denis' answer is very good, I won't re-iterate his points.
>
> Thanks. And also thanks for pointing out the Hardlinks thing, I
> over-read the "lots of small files" part in Velkjos Mail.
>
> Anyway, I have some comparison data. I have a backup server that saves
> data from 5 other server at our hosting company using rsnapshot. The
> backups are kept for 14 days.
>
> rsnapshot:
>
> The Backup has 186GB. 51 GB for the full backup (daily.0) and
> about 11GB for each incremental backup (daily.1 - daily.13). The backup
> includes typical small webserver files but also big logfiles and two
> ZOPE Databases (ZEOs) with about 5GB each.
>
> bup:
>
> I imported them with "bup import-rsnapshot", the overall size is 15GB
> (for all 14 days) which is quite amazing. Anyway the lack of a
> possibility to delete old backup versions is (for me) a major drawback.
> What I liked was the possibility to mount the Backup with FUSE. After
> mounting the Backup one can access every backup generation as normal
> files. Each generation has its own folder with a timestamp. The files
> inside the backup have no metadata (timestamp is always 1.1.1970, etc.).
>
> The only way I can think of at the moment to get rid of old backup
> generations using bup is to mount (FUSE) the backup restore all backup
> generations you want to keep to an additional drive, delete the bup git
> repository, create a new one and backup the restore again. Of course
> this might take a lot of additional space on you disk for the
> (temporary) restore which might not be available. If any has some
> better approach I would love to hear.
>
> obnam:
>
> With obnam I made a backup of daily.0 (51GB). There was nearly no
> reduction in the size for the first backup run (47GB). The next backup
> run (one day later, which creates 11GB new data with rsnapshot) has only
> added a few MB and therefore was pretty fast.
>
> The repository approach of obnam comes very handy. You can pull or push
> backups to the repository server and can access the backups from any
> other machine (if you have SSH access). Configuration is not necessary
> but a small config containing some default parameters comes in handy:
>
> [config]
> repository = sftp://192.168.1.10/backup/obnam/
> log = /var/log/obnam.log
> log-level = warning
> client-name = dx
>
> If you now run "obnam backup /var/www" the backup of /var/www will be
> pushed to the repository. obnam locks the repository for the client so
> one cannot accidentally run two backups of the same host (client) at
> the same time. Running several backups of different hosts is no problem.
>
> During a backup run obnam makes "snapshots" every few 100MB so if the
> backup fails (e.g. disconnect from the repository server) the backup
> can be resumed from the last snapshot.
>
> A nice feature is some kind of built in nagios plugin:
>
> obnam nagios-last-backup-age --warn-age=1 --client=dx
> OK: backup is recent. last backup was 2012-09-12 10:05:47.
>
> obnam nagios-last-backup-age --warn-age=1 --client=backup
> WARNING: backup is old. last backup was 2012-09-11 18:03:43.
>
> obnam nagios-last-backup-age --client=cat --critical-age=1
> CRITICAL: backup is old. last backup was 2012-09-11 17:01:23.
>
> The restore is a bit more complex, as there is (at the moment) no FUSE
> filesystem available for obnam. Instead you need to know the name of
> the file/folder and in which backup generation your file/folder exists.
>
> "obnam generations" shows all available backups:
>
> 101 2012-09-11 18:01:07 .. 2012-09-11 18:02:10 (26474 files,
> 8598496965 bytes)
> 108 2012-09-12 10:05:47 .. 2012-09-12 10:06:36 (26474 files,
> 8598500897 bytes)
>
> Then you can use "obnam ls --generation=101" to show the files.
>
> rdiff-backup:
>
> If have no real comparison data for rdiff-backup but I expect similar
> results as with obnam (about 50GB for the first backup, only several MB
> for each following daily backup).
>
> rdiff-backup can (like bup) mount the backup (all generations) using
> FUSE.
>
> Best regards
> Denis Witt
This is excellent comparison. Taking everything in consideration, I
thing I narrowed my choice to obnam, rdiff-backup and rsnapshot. I'll
try them all and see how they behave on my machine.
obnam and rdiff-backup seems to use less space, but I also like very
clear representation of backups on rsnapshot. But during few days of
testing each of them I'll know what to use.
I also stumbled upon this one: http://rbackup.lescigales.org/
Best regards,
Veljko
Reply to: