Re: reading an empty directory after reboot is very slow
On 2015-04-21 11:05:58 -0500, David Wright wrote:
> Quoting Vincent Lefevre (vincent@vinc17.net):
> > On 2015-04-20 13:04:41 -0500, David Wright wrote:
> > > Quoting Vincent Lefevre (vincent@vinc17.net):
> > > > But with the current solution (no automatic moving of an entry), you
> > > > can't miss an entry that hasn't been removed.
> [...]
> > > ...so if you happen to be reading the entry for file5 at the
> > > time I typed mv, you'll get the entry for file4 twice, under
> > > two different names. (Or the opposite.)
> >
> > OK, so, if the rename(2) system call can reorder the entries (this is
> > not quite clear because one doesn't see the empty entries here),
>
> No, and you wouldn't *normally* see them with readdir, I'd suppose.
But this matters for the implementation in the kernel.
> > then
> > there is already a problem with the file system. Getting an entry
> > twice under different names is not much a problem, IMHO, because one
> > can look at the inode number; there's a race condition, but at worst,
> > one can just miss a *new* inode (whose number has been reassigned).
> > Missing an existing entry is a problem.
>
> ...easily demonstrated with
>
> ~ $ for j in 1 2 3 4 5 6 ; do mkdir /tmp/testdir/file$j ; done
> ~ $ ls -lU /tmp/testdir
> total 24
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file1
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file4
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file5
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file6
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file2
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file3
> ~ $ mv -i /tmp/testdir/file3 /tmp/testdir/file3file3file3file3file3file3file3file3file3file3file3file3file3
> ~ $ ls -lU /tmp/testdir
> total 24
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file1
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file4
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file5
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file3file3file3file3file3file3file3file3file3file3file3file3file3
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file6
> drwxr-x--- 2 david david 4096 Apr 21 10:58 file2
> ~ $
>
> where file3 goes AWOL.
You haven't demonstrated anything. If you have before the mv:
0: file1
1: file4
2: file5
3: file6
4: file2
5: file3
and after the mv:
0: file1
1: file4
2: file5
3: [empty]
4: [empty]
5: file3file3file3file3file3file3file3file3file3file3file3file3file3
6: file6
7: file2
there's no problem. What actually needs to be done is a real test
using readdir.
Any idea of the algorithm to choose the directory entries? The fact
that the files are not ordered initially is unintuitive.
> > What do the backup systems do?
>
> I don't know. Lock the directory and slurp it (if it's not too big),
> otherwise check the modification time before and after reading it,
> and reread it if necessary, maybe...
The modification time mustn't be used since it can be changed by some
tools, e.g. during unarchiving or decompressing. The ctime could be
OK, but re-reads can introduce endless loops for directories that are
constantly modified.
--
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Reply to: