Re: Backup solutions without reinventig the wheel these days

Thank you Sarunas for the graphs, it is most illuminating. That's pretty much how I imagine a storage server should behave.

Atomic snapshots are very appealing indeed. In my case cp -lr or --link-dest solutions would take a long time as the FS hierarchy is both very broad and deep.

So far it seems I will go with backupninja with simple rsync transport and a few custom shell scripts to create and delete btrfs snapshots on the remote server. This means I will need root access on the backup server, but I can restrict that with authorized_keys or sudo.

Is there some other framework similar to backupninja that provides the common boilerplate code for the cron + command + mail report paradigma?

I also like the various backupninja helper scripts.

I was thinking about Bacula, but I'm not sure if it has block-level deduplication, from the docks it seems it has file-level dedup. It seems like a bit of an overkill for my case.

On Wed, Oct 21, 2015 at 5:07 PM, Sarunas Burdulis <sarunas@math.dartmouth.edu> wrote:

On 10/21/2015 10:47 AM, Ondřej Grover wrote:
> I could try to request a RAM upgrade (not super easy in uni environment,
> but doable), but it goes against my understanding of what a storage
> server is. In my book, a storage server needs fast IO capabilities
> (storage drives and NIC) and possibly CPU for compression. Deduplication
> is the only reason I could think of higher RAM requirements on a storage
> server.

This morning I added a 'memory' plugin to the Munin node on the server I
described. Here is a snaphot so far, for what it's worth. The bump at
9:50 is an rsync from one of the biggest filesystems backed-up (~600GiB,
about a million files to sync).

https://math.dartmouth.edu/owncloud/index.php/s/E3VYxhWlCqUusn4

> Sarunas, did you have good experiences with actually using the backups
> for restoring?

Whenever I needed them, backup copies were available. This is more of an
archival storage. The main point for using btrfs was instantaneous
snapshot creation (and removal of older ones, something that takes
'forever' in rsync --link-dest scenario, using rsnapshot, for example).

Sarunas

> On Wed, Oct 21, 2015 at 4:28 PM, Sarunas Burdulis
> <sarunas@math.dartmouth.edu <mailto:sarunas@math.dartmouth.edu>> wrote:
>
> On 10/21/2015 05:32 AM, Ondřej Grover wrote:
> > [...]
> > Your btrfs setup sounds interesting, Sarunas. Are you running those
> > btrfs volumes on Jessie? Any stability issues? Do you think it could
> > work with just 1GB RAM + 1 GB swap? I was thinking about doing
> > out-of-band deduplication to conserve RAM. Do you use both compression
> > and deduplication (file or block level)? I heard they don't play nicely
> > together. Could you share your mount options?
>
> Our backup server is actually running Linux 3.13.0-66-generic, Btrfs
> v3.12 — Ubuntu 14.04.3 LTS, which is behind Jessie in software versions.
> It has been very stable — for the past too years it's running as a
> headless box in an off-site location.
>
> We don't use compression or deduplication. In-band dedup is not in btrfs
> 3.*, and I haven't tried any out-of-band tools. Nothing particular in
> Btrfs mount, the only non-default options in 'noatime':
>
> LABEL=BACKUPS /backups btrfs noatime 0 0
>
> 1GB RAM sounds low by today's habits. That doesn't mean it won't work,
> but just begs for RAM upgrade.
>
> Sarunas
>
> > On Tue, Oct 20, 2015 at 8:35 PM, Sarunas Burdulis
> > <sarunas@math.dartmouth.edu <mailto:sarunas@math.dartmouth.edu>
> <mailto:sarunas@math.dartmouth.edu

> <mailto:sarunas@math.dartmouth.edu>>> wrote:
> >
> > On 10/20/2015 12:57 PM, Ondřej Grover wrote:
> >> Hello,
> >
> >> I'm looking for recommendations for backup solutions that don't
> reinvent
> >> the wheel and are reliable and used. I want to backup two servers
> to a
> >> backup server. The main data content is several hundred GB in
> many very
> >> small files.
> >
> >> [...]
> >
> >> I've also looked at the new kids on the block like obnam, attick and
> >> borgbackup. They look interesting, but I prefer time-tested SW
> for backups.
> >> After realizing that these new backup programs pretty much try to
> >> replicate features of btrfs or ZFS (incremental snapshots,
> block-level
> >> compression and deduplication) I started thinking that I could
> perhaps
> >> just send the data to the backup server via rsync and save them to a
> >> btrfs or ZFS (but the backup server may not have enough RAM for
> ZFS) and
> >> create daily snaphosts on the server. If memory will permit (if I
> >> optimize it), I'd go with ZFS as it should be more reliable. Does
> >> anybody use such a solution?
> >
> > Yes, we do use pretty much the setup you describe, i.e. rsync push to
> > backup server, btrfs volume with daily snapshot subvolumes on the
> > server. This particular setup has been in use for more than two
> years by
> > now, and is working well with 8-10 servers (totals for all backed-up
> > filesystems is about 1.2TiB/2.5M files of varying size; transferred
> > daily delta is of course just a fraction of this). The backup
> server is
> > a rather modest self-built Mini-ITX Core i5 with 8GiB RAM and 6TiB
> > hardware (3ware) RAID-10 storage volume.
> >
> > Using btrfs volume in btrfs-raid configuration (raid-1 for data and
> > metadata) would provide protection against data bit-rot. We use
> several
> > such volumes on some of our severs, though not in combination with
> btrfs
> > snapshot subvolumes.
> >
> > --
> > Sarunas Burdulis
> > http://math.dartmouth.edu/~sarunas
>
>

--
Sarunas Burdulis
Systems Administrator
Department of Mathematics, Dartmouth College
http://math.dartmouth.edu/~sarunas