[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: reading an empty directory after reboot is very slow

Quoting Vincent Lefevre (vincent@vinc17.net):
> On 2015-04-24 19:41:03 -0500, David Wright wrote:
> > Well, that's what I thought, because of the cacheing. But Nicolas
> > asked me to try using thousands of files, and so here we are,
> > ie, your "new test":
> > 
> > ~ $ for j in `seq 10000` ; do mkdir /tmp/testdir/file$j ; done
> > ~ $ ./a.out /tmp/testdir/ > lsout1
> > ~ $ ./a.out /tmp/testdir/ > lsout2 ← here I renamed file2621
> > ~ $ ./a.out /tmp/testdir/ > lsout3
> > ~ $ wc lsout*
> >  10003  40009 237834 lsout1
> >  10002  40005 237809 lsout2 ← missing entry
> >  10003  40009 237890 lsout3
> >  30008 120023 713533 total
> > ~ $ 
> [...]
> > So we have a file gone AWOL because it was renamed during this
> > program's execution. (The I numbers here come from dirent;
> > there's no call of stat.)
> So, I would say that this is a bug. POSIX says in
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/readdir.html

It may well be. But I'm just presenting facts about the
filesystems that I assumed you were using when you posted
https://lists.debian.org/debian-user/2015/04/msg00651.html ,
not the theory of a POSIX-compliant system.

>   If a file is removed from or added to the directory after the most
>   recent call to opendir() or rewinddir(), whether a subsequent call
>   to readdir() returns an entry for that file is unspecified.
> But when a file is renamed, it is nowhere said that the file is
> removed from and/or added to the directory. The POSIX spec just
> says that the file is "renamed".
> [...]
> > You said that without "Bob's move", you couldn't miss an entry that
> > hasn't been removed.
> > 
> > I've demonstrated that you can.
> Because of a *bug*. With a conforming implementation, you can't.
> (And if a renamed file were regarded as an entry that was removed,
> then this would mean that your example would not be correct, because
> this would not be "an entry that hasn't been removed".)

It may be a bug. It may be a misfeature in the interests of
performance. I don't know. When writing big programs, decisions have
to be taken.

But my demonstration stands. If you want to reject my observations by
bringing in POSIX-compliance as a precondition for my tests, that
would be both inconsiderate (ie you'd be telling me I've wasted my
time doing all the tests that you encouraged me to perform), and a
post disputation argument.

This thread would have taken a different course if you'd posted
"But with a POSIX-compliant filesystem, as ext3/4 is supposed to be,
you can't miss an entry that hasn't been removed." Then I could have
looked at the docs and skipped all the tests.


Reply to: