Re: Safe File Update (atomic)

[Henrique de Moraes Holschuh <hmh@debian.org>]

On Wed, 29 Dec 2010, Olaf van der Spek wrote:
> Writing a temp file, fsync, rename is often proposed. However, the

It is:
  write temp file (in same directory as file to be replaced), fsync temp
  file[1], rename (atomic), fsync directory[2].

[1] Makes sure file data has been commited to backend device before
    the metadata update

[2] Makes sure the metadata has been commited to permantent storage.
    Can often be ignored when you don't really care to know you will
    get the new contents (as opposed to the old contents) in case of
    a crash.  MTAs and spools, for example, MUST do it.

Which steps you can skip is filesystem-options/filesystem/
kernel-version/kernel dependent.  When the rename acts as a barrier, [1]
can be skipped, for example.  Tracking this is a losing proposition.

If we could use some syscall to make [1] into a simple barrier request
(guaranteed to degrade to fsync if barriers are not operating), it would
be better performance-wise.  This is what one should request of libc and
the kernels with a non-zero chance of getting it implemented (in fact,
it might even already exist).

> I've brought this up on linux-fsdevel and linux-ext4 but they (Ted)
> claim those exceptions aren't really a problem.

Indeed they are not.  Code has been dealing with them for years.  You
name the temp file properly, and teach your program to clean old ones up
*safely* (see vim swap file handling for an example) when it starts.

vim is a good example: nobody gets surprised by vim swap-files left over
when vim/computer crashes. And vim will do something smart with them if
it finds them in the current directory when it is started.

BTW: safely removing a file is also tricky.  AFAIK, one must open it RW,
in exclusive mode. stat it by fd and check whether it is what one
expects (regular file, ownership).  unlink it by fd.  close the fd.

> Is there a code snippet or lib function that handles this properly?

I don't know.  I'd be interested in the answer, though :-)

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh