[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Safe File Update (atomic)


Thanks for the reply and detailed analysis.

> Which gets me back to the question of use cases.  When are we going to
> be using this monster?  For many use cases, where the original reason

Where "implicit rollbacks" are desireable, I suppose.  It is incompatible
with edit-in-place, anyway.  Which asks for all the fsyncs on the link
thing.  Anyone that wants something different is welcome to do it the old
way IMHO.

The first part (get an unlinked fd) is useful without fsyncs or any
guarantees for temp files.

> by the kernel.  But if you use make the system call synchronous, now
> there's no performance advantage over simply doing the fsync() and
> rename() in userspace.  And if we do this using O_ATOMIC, or your

I understand this is far more about "ease of use" (read: more difficult to
misuse) than "much higher performance".

> 1) You care about data loss in the case of power failure, but not in
> the case of hard drive or storage failure, *AND* you are writing tons
> and tons of tiny 3-4 byte files and so you are worried about
> performance because you're doing something insane with large number of
> small files.

That usage pattern cannot be made both safe and fast outside of a full-blown
ACID database, so lets skip it.

> 2) You are specifically worried about the case where you are replacing
> the contents of a file that is owned by different uid than the user
> doing the data file update, in a safe way where you don't want a
> partially written file to replace the old, complete file, *AND* you
> care about the file's ownership after the data update.

I am not sure about the file ownership, but this is the useful usecase IMO.

> 3) You care about the temp file used by the userspace library, or
> application which is doing the write temp file, fsync(), rename()
> scheme, being automatically deleted in case of a system crash or a
> process getting sent an uncatchable signal and getting terminated.

This is always useful, as well.

> Is it worth it?  I'd say no; and suggest that someone who really cares
> should create a userspace application helper library first, since
> you'll need it as a fallback for the cases listed above where this
> scheme won't work.  (Even if you do the fallback in the kernel, you'll
> still need userspace fallback for non-Linux systems, and for when the
> application is run on an older Linux kernel that doesn't have all of
> this O_ATOMIC or link/unlink magic).

That's what I suggested, as well.

> The reality is we've lived without this capability in Unix and Linux
> system for something like three decades.  I suspect we can live

But not very well.  And the usage patterns of *nix systems have changed in
the last decade.

  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Reply to: