On 10/21/2015 05:32 AM, Ondřej Grover wrote:
> [...]
> Your btrfs setup sounds interesting, Sarunas. Are you running those
> btrfs volumes on Jessie? Any stability issues? Do you think it could
> work with just 1GB RAM + 1 GB swap? I was thinking about doing
> out-of-band deduplication to conserve RAM. Do you use both compression
> and deduplication (file or block level)? I heard they don't play nicely
> together. Could you share your mount options?
Our backup server is actually running Linux 3.13.0-66-generic, Btrfs
v3.12 — Ubuntu 14.04.3 LTS, which is behind Jessie in software versions.
It has been very stable — for the past too years it's running as a
headless box in an off-site location.
We don't use compression or deduplication. In-band dedup is not in btrfs
3.*, and I haven't tried any out-of-band tools. Nothing particular in
Btrfs mount, the only non-default options in 'noatime':
LABEL=BACKUPS /backups btrfs noatime 0 0
1GB RAM sounds low by today's habits. That doesn't mean it won't work,
but just begs for RAM upgrade.
Sarunas
> On Tue, Oct 20, 2015 at 8:35 PM, Sarunas Burdulis
> <sarunas@math.dartmouth.edu <mailto:sarunas@math.dartmouth.edu>> wrote:
>
> On 10/20/2015 12:57 PM, Ondřej Grover wrote:
>> Hello,
>
>> I'm looking for recommendations for backup solutions that don't reinvent
>> the wheel and are reliable and used. I want to backup two servers to a
>> backup server. The main data content is several hundred GB in many very
>> small files.
>
>> [...]
>
>> I've also looked at the new kids on the block like obnam, attick and
>> borgbackup. They look interesting, but I prefer time-tested SW for backups.
>> After realizing that these new backup programs pretty much try to
>> replicate features of btrfs or ZFS (incremental snapshots, block-level
>> compression and deduplication) I started thinking that I could perhaps
>> just send the data to the backup server via rsync and save them to a
>> btrfs or ZFS (but the backup server may not have enough RAM for ZFS) and
>> create daily snaphosts on the server. If memory will permit (if I
>> optimize it), I'd go with ZFS as it should be more reliable. Does
>> anybody use such a solution?
>
> Yes, we do use pretty much the setup you describe, i.e. rsync push to
> backup server, btrfs volume with daily snapshot subvolumes on the
> server. This particular setup has been in use for more than two years by
> now, and is working well with 8-10 servers (totals for all backed-up
> filesystems is about 1.2TiB/2.5M files of varying size; transferred
> daily delta is of course just a fraction of this). The backup server is
> a rather modest self-built Mini-ITX Core i5 with 8GiB RAM and 6TiB
> hardware (3ware) RAID-10 storage volume.
>
> Using btrfs volume in btrfs-raid configuration (raid-1 for data and
> metadata) would provide protection against data bit-rot. We use several
> such volumes on some of our severs, though not in combination with btrfs
> snapshot subvolumes.
>