[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Live Fille System Backup

Hash: SHA1

On Thu, May 04, 2017 at 04:17:46PM -0700, Sergei G wrote:
> I am running Raspberry PI and I would like to dump full file system without
> shutting down the system.  One machine runs nginx and another runs
> PostgreSQL.  I have had a good success with FreeBSD and dump software,
> because it is part of the OS and core team maintains it.

Rsync. Hands down.

That said...

there are some thigs you might want to take into account.

 - A solution like rsync alone won't ever solve the "skew" problem
   (i.e. the file system moves while the backup is being taken).
   Depending on how sensitive your "world" is to that, this might
   be a problem or not. "Typically" (famous last words, I know)
   it ain't for me, especially if I take care of taking a "second
   rsync", which will be very fast. If that isn't enough, you might
   like to "snapshot" with LVM or overlayfs and back-up your
   "frozen" (and thus consistent) snapshot, while life throbs on
   on your "real" file system branch.

 - PostgreSQL. Conventional wisdom says you shouldn't back up a
   running database. Nevertheless, you can do that with PostgreSQL,
   if you take care of orchestrating pg_start_backup, pg_stop_backup
   and the right copying of WALs [1]. This sounds complicated, is
   not: your (homegrown or acquired) backup scripts just have to
   provide a "before" and an "after" hook (if they don't, claim your
   money back :) where to enlist the scripts to steer your database(s).

   Having an old-fashioned pg_dump around which isn't too old won't
   hurt, especially if you want to be able to restore to a significantly
   different architecture/ PostgreSQL version.

Things rsync won't be very good at:

  - huge backups (several terabytes or more; millions of files)
    My current backup is around 48 GB and 650K files (not much),
    and the rsync approach (organic, homegrown :) doesn't break
    a sweat, with fairly modest hardware. YMMV. I've seen a photo
    archive (~20TB) where the buildup of the file list would have
    taken rsync short of two days -- that won't work, obviously
    (to be fair, rsync has gotten better in this department as
    of late, and the shop in question has a faible for "seasoned"
    software versions).

 - whatever I forgot: please chime in!


[1] https://www.postgresql.org/docs/current/static/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP
- -- tomás
Version: GnuPG v1.4.12 (GNU/Linux)


Reply to: