Re: reading an empty directory after reboot is very slow
Quoting Vincent Lefevre (vincent@vinc17.net):
> On 2015-04-24 19:41:03 -0500, David Wright wrote:
> > Well, that's what I thought, because of the cacheing. But Nicolas
> > asked me to try using thousands of files, and so here we are,
> > ie, your "new test":
> >
> > ~ $ for j in `seq 10000` ; do mkdir /tmp/testdir/file$j ; done
> > ~ $ ./a.out /tmp/testdir/ > lsout1
> > ~ $ ./a.out /tmp/testdir/ > lsout2 ← here I renamed file2621
> > ~ $ ./a.out /tmp/testdir/ > lsout3
> > ~ $ wc lsout*
> > 10003 40009 237834 lsout1
> > 10002 40005 237809 lsout2 ← missing entry
> > 10003 40009 237890 lsout3
> > 30008 120023 713533 total
> > ~ $
> [...]
> > So we have a file gone AWOL because it was renamed during this
> > program's execution. (The I numbers here come from dirent;
> > there's no call of stat.)
>
> So, I would say that this is a bug. POSIX says in
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/readdir.html
It may well be. But I'm just presenting facts about the
filesystems that I assumed you were using when you posted
https://lists.debian.org/debian-user/2015/04/msg00651.html ,
not the theory of a POSIX-compliant system.
> If a file is removed from or added to the directory after the most
> recent call to opendir() or rewinddir(), whether a subsequent call
> to readdir() returns an entry for that file is unspecified.
>
> But when a file is renamed, it is nowhere said that the file is
> removed from and/or added to the directory. The POSIX spec just
> says that the file is "renamed".
>
> [...]
> > You said that without "Bob's move", you couldn't miss an entry that
> > hasn't been removed.
> >
> > I've demonstrated that you can.
>
> Because of a *bug*. With a conforming implementation, you can't.
>
> (And if a renamed file were regarded as an entry that was removed,
> then this would mean that your example would not be correct, because
> this would not be "an entry that hasn't been removed".)
It may be a bug. It may be a misfeature in the interests of
performance. I don't know. When writing big programs, decisions have
to be taken.
But my demonstration stands. If you want to reject my observations by
bringing in POSIX-compliance as a precondition for my tests, that
would be both inconsiderate (ie you'd be telling me I've wasted my
time doing all the tests that you encouraged me to perform), and a
post disputation argument.
This thread would have taken a different course if you'd posted
"But with a POSIX-compliant filesystem, as ext3/4 is supposed to be,
you can't miss an entry that hasn't been removed." Then I could have
looked at the docs and skipped all the tests.
Cheers,
David.
Reply to: