[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Safe File Update (atomic)

On Thu, Dec 30, 2010 at 12:46 PM, Henrique de Moraes Holschuh
<hmh@debian.org> wrote:
>  write temp file (in same directory as file to be replaced), fsync temp

What if the target name is actually a symlink? To a different volume?
What if you're not allowed to create a file in that dir.

> If we could use some syscall to make [1] into a simple barrier request
> (guaranteed to degrade to fsync if barriers are not operating), it would
> be better performance-wise.  This is what one should request of libc and
> the kernels with a non-zero chance of getting it implemented (in fact,
> it might even already exist).

My proposal was O_ATOMIC:
// begin transaction
open(fname, O_ATOMIC | O_TRUNC);
write; // 0+ times

Seems like the ideal API from the app's point of view.

>> I've brought this up on linux-fsdevel and linux-ext4 but they (Ted)
>> claim those exceptions aren't really a problem.
> Indeed they are not.  Code has been dealing with them for years.  You

Code has been wrong for years to, based on the reason reports about
file corruption with ext4.

> name the temp file properly, and teach your program to clean old ones up
> *safely* (see vim swap file handling for an example) when it starts.

What about restoring meta-data? File-owner?

> vim is a good example: nobody gets surprised by vim swap-files left over
> when vim/computer crashes. And vim will do something smart with them if
> it finds them in the current directory when it is started.

I'm sure the vim code is far from trivial. I think this complexity is
part of the reason most apps don't bother.

> BTW: safely removing a file is also tricky.  AFAIK, one must open it RW,
> in exclusive mode. stat it by fd and check whether it is what one

Exclusive mode? Linux doesn't know about mandatory locking (AFAIK).

> expects (regular file, ownership).  unlink it by fd.  close the fd.
>> Is there a code snippet or lib function that handles this properly?
> I don't know.  I'd be interested in the answer, though :-)

I'll ask glibc.


Reply to: