Re: Safe File Update (atomic)
On Sun, Jan 02, 2011 at 04:14:15PM +0100, Olaf van der Spek wrote:
>
> Last time you ignored my response, but let's try again.
> The implementation would be comparable to using a temp file, so
> there's no need to keep 2 g in memory.
> Write the 2 g to disk, wait one day, append the 1 k, fsync, update inode.
Write the 2g to disk *where*? Some random assigned blocks? And using
*what* to keep track of the where to find all of the metadata blocks?
That information is normally stored in the inode, but you don't want
to touch it. So we need to store it someplace, and you haven't
specified where. Some alternate universe? Another inode, which is
only tied to that file descriptor? That's *possible*, but it's (a)
not at all trivial, and (b) won't work for all file systems. It
definitely won't work for FAT based file systems, so your blithe, "oh,
just emulate it in the kernel" is rather laughable.
If you think it's so easy, *you* go implement it.
> > How exactly do the semantics for O_ATOMIC work?
> >
> > And given at the momment ***zero*** file systems implement O_ATOMIC,
> > what should an application do as a fallback? And given that it is
>
> Fallback could be implement in the kernel or in userland. Using rename
> as a fallback sounds reasonable. Implementations could switch to
> O_ATOMIC when available.
Using rename as a fallback means exposing random temp file names into
the directory. Which could conflict with files that the userspace
might want to create. It could be done, but again, it's an awful lot
of complexity to shove into the kernel.
> > highly unlikely this could ever be implemented for various file
> > systems including NFS, I'll observe this won't really reduce
> > application complexity, since you'll always need to have a fallback
> > for file systems and kernels that don't support O_ATOMIC.
>
> I don't see a reason why this couldn't be implemented by NFS.
Try it; it should become obvious fairly quickly. Or just go read the
NFS protocol specifications.....
> As you've said yourself, a lot of apps don't get this right. Why not?
> Because the safe way is much more complex than the unsafe way. APIs
> should be easy to use right and hard to misuse. With O_ATOMIC, I feel
> this is the case. Without, it's the opposite and the consequences are
> obvious. There shouldn't be a tradeoff between safety and potential
> problems.
Application programmers have in the past been unwilling to change
their applications. If they are willing to change their applications,
they can just as easily use a userspace library, or use fsync() and
rename() properly. If they aren't willing to change their programs
and recompile (and the last time we've been around this block, they
weren't; they just blamed the file system), asking them to use
O_ATOMIC probably won't work, given the portiability issues.
> > And of course, Olaf isn't actually offerring to implement this
> > hypothetical O_ATOMIC. Oh, no! He's just petulently demanding it,
> > even though he can't give us any concrete use cases where this would
> > actually be a huge win over a userspace "safe-write" library that
> > properly uses fsync() and rename().
>
> Not true. I've asked (you) for just such a lib, but I'm still waiting
> for an answer.
Pay someone enough money, and they'll write you the library. Whining
about it petulently and expecting someone else to write it is probably
not going to work.
Quite frankly, if you're competent enough to use it, you should be
able to write such a library yourself. If you aren't going to be
using it yourself, they why are you wasting everyone's time on this?
- Ted
Reply to: