Re: Safe File Update (atomic)
On Mon, Jan 03, 2011 at 09:49:40AM -0200, Henrique de Moraes Holschuh wrote:
>
> > 1) You care about data loss in the case of power failure, but not in
> > the case of hard drive or storage failure, *AND* you are writing tons
> > and tons of tiny 3-4 byte files and so you are worried about
> > performance because you're doing something insane with large number of
> > small files.
>
> That usage pattern cannot be made both safe and fast outside of a full-blown
> ACID database, so lets skip it.
Agreed.
>
> > 2) You are specifically worried about the case where you are replacing
> > the contents of a file that is owned by different uid than the user
> > doing the data file update, in a safe way where you don't want a
> > partially written file to replace the old, complete file, *AND* you
> > care about the file's ownership after the data update.
>
> I am not sure about the file ownership, but this is the useful usecase IMO.
But if you don't care about file ownership, then you can do the write
a temp file, fsync, and rename trick. If it's about "ease of use", as
you suggest, a userspace library solves that problem. It's *only* if
you care about the file ownership remaining the same that (2) comes
into play.
> > 3) You care about the temp file used by the userspace library, or
> > application which is doing the write temp file, fsync(), rename()
> > scheme, being automatically deleted in case of a system crash or a
> > process getting sent an uncatchable signal and getting terminated.
>
> This is always useful, as well.
.... and (3) is the recovery after a power failure/crash scenario
If you don't care about the file ownership issue, then recovering
after a powerfailure/crash is the last remaining case --- and you
could solve this by creating a file with an mktemp-style name in a
mode 1777 directory, where the contents of the file contains the temp
file name to be deleted by an init.d script. This could be done in
the userspace library, and if you crash after the rename, but before
you have a chance to delete the file containing the
temp-filename-to-be-deleted, that's not a problem, since the init.d
file will find no file with that name to be deleted, and then
continue.
Hence, all of these problems can be solved in userspace, with a
userspace library, with the exception of the file ownership issue,
which you've admitted may not be all that critical.
> > Is it worth it? I'd say no; and suggest that someone who really cares
> > should create a userspace application helper library first, since
> > you'll need it as a fallback for the cases listed above where this
> > scheme won't work. (Even if you do the fallback in the kernel, you'll
> > still need userspace fallback for non-Linux systems, and for when the
> > application is run on an older Linux kernel that doesn't have all of
> > this O_ATOMIC or link/unlink magic).
>
> That's what I suggested, as well.
Then we're in agreement. :-)
- Ted
Reply to: