[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Live Fille System Backup

We have to remember that things are actually worse at the application consistency level.  An application may think that it has committed its write, but the file system has not written all the bytes to the disk yet.  

File system crash consistency is a sibling of the backup topic.  Here is relevant article I run into a some time ago https://blog.acolyer.org/2017/03/15/application-crash-consistency-and-performance-with-ccfs/

According to link above Ext4 FS suffers from these inconsistencies.

On Sun, May 7, 2017 at 7:56 AM, <tomas@tuxteam.de> wrote:
Hash: SHA1

On Sun, May 07, 2017 at 08:42:36AM -0400, rhkramer@gmail.com wrote:
> On Sunday, May 07, 2017 05:47:20 AM tomas@tuxteam.de wrote:
> <good stuff elided>
> > Otherwise, "on line" backup is simply not an option.


> I guess the way that comes to mind, maybe somewhat implied by what you've
> written (or at least inferred by me ;-) would be to take down one or more of
> the servers at a time, make static backups of those, then restore those to
> service and repeat until all have been backed up, very likely repeating that
> continuously.

"Taking down" as in "stopping all relevant applications" (shutting down
the whole thing would be a superset of that, of course ;)


> Oh, ok, I guess (well, I know RAID isn't really for backup, it's for uptime as
> is often stated on this and other lists), but something like RAID where
> relevant data is written to more than one disk and / or storage "farm", one
> being the "in service" device (to serve data to the app in "real time") and
> one or more just recording data as backup.

This is a variant on the snapshot pattern. You take a snapshot of the
file system and back up that. You are guaranteed that the *file system*
backup is consistent (well, modulo bugs and glitches), but since the
applications don't have a clue what's going on, application state
might still be inconsistent, unless they go out of their way to keep
a consistent state on-disk all of the time (in the above scenario
of shutdown, we assumed that the application leaves a consistent
state after being shut down, which is a reasonable assumption, I'd

That said, and as you can see in the PostgreSQL example, with a little
help of your application, the snapshot thing works (in the case of
PostgreSQL things are much more relaxed and you can usually go with
plain rsync, for example, without resorting to snapshot).

All in all I'm pretty happy with a plain, straight rsync-style backup
for my workstation. The probability of a busted backup is so low that
I fear more for my backup medium going sour.

On a high-churn machine with very valuable data things might look
differently, though...

- -- tomás
Version: GnuPG v1.4.12 (GNU/Linux)


Reply to: