[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Safe File Update (atomic)

On Sun, Jan 2, 2011 at 7:55 PM, Adam Borowski <kilobyte@angband.pl> wrote:
> Note that on the other side of the fence there's something called TxF


> And what if you're changing one byte inside a 50 GB file?
> I see an easy implementation on btrfs/ocfs2 (assuming no other writers),
> but on ext3/4, that'd be tricky.

My proposal is explicitly only for complete file data updates.

>> > what should an application do as a fallback?  And given that it is
>> Fallback could be implement in the kernel or in userland. Using rename
>> as a fallback sounds reasonable. Implementations could switch to
>> O_ATOMIC when available.
> For large files using reflink (currently implemented as fs-specific ioctls)
> can ensure performance.  It can give you anything but the abuse for
> preserving owner (ie, the topic of this thread).  To get that, you'd need
> in-kernel support, but for example http://lwn.net/Articles/331808/ proposes
> an API which is just a thin wrapper over existing functionality in multiple
> filesystems.  It basically duplicates an inode, preserving all current
> attributes but making any new writes CoWed.  If you make the old one
> immutable, you get the TxF semantics (mandatory write lock), if you don't,
> you'll get the mentioned above "one of the updates will win" data loss.

Data loss? If you overwrite a file, losing the old contents isn't data loss.

>> > And what are the use cases where this really makes sense?  Will people
>> Lots of files are written in one go. They could all use this interface.
> I don't see how O_ATOMIC helps there.  TxF transactions would work (all
> writes either succeed together or none does), but O_ATOMIC can't do more
> than one file.

I mean that each app that writes a file in one go could use the O_ATOMIC API.
Extending O_ATOMIC to support multiple files seems simple too by using
a vector variant of close.

> Uhm, but you didn't answer the question.  These two use cases Ted Tso
> mentioned are certainly not worth the complexity of in-kernel support,
> O_ATOMIC doesn't bring other goodies, and the rest can be done by an
> userspace library which is indeed a good idea.

Someone is working on such a lib, let's see the code complexity and
exceptions it has.

>> Not true. I've asked (you) for just such a lib, but I'm still waiting
>> for an answer.
> Shachar Shemesh is already working on it, when he finishes, Ted Tso will
> point out what's wrong in it (if something is).  What else do you need?

Don't know yet. Let's wait for that lib.

>> Why would anyone work on an implementation if there's no agreement about it?
> Because one implementation after research is better than many naive and
> possibly wrong ones.

True, but it'd still be nice to have some agreement before doing all
that hard work.


Reply to: