Re: Bug#605009: serious performance regression with ext4
On Mon, Nov 29, 2010 at 02:16:02PM +0100, Raphael Hertzog wrote:
>
> It means we don't need to keep it in RAM since we're not going to
> read/modifiy it again in the near future. Thus the writeback can be
> started right now since delaying it will not save us anything.
>
> At least that's the way I understand the situation.
Yes, that's correct. The fadvise() will do two things; it will start
the writeback, and also make these memory pages be the most likely to
be discarded. This last might or might not be a good thing. If you
are installing a large number of packages, discarding will avoid more
useful things from being discard, and might help the interactive
"feel" of the machine while the install is going on in the background.
OTOH, if you are only installing one package, it might cause some file
that will be needed by the postinstall script to be pushed out of the
page cache prematurely.
So the fadvise() does the same thing as SYNC_FILE_RANGE_WRITE, which
is to say, start an asynchronous writeback of the pages in the file.
It will not do a SYNC_FILE_RANGE_WAIT_BEFORE, which assures that the
writebacks are complete before attempting to start doing the
fdatasync().
> > Put another way: if this works now, is it likely to continue to work?
>
> Well, it will always work (the code is unlikely to introduce failures),
> but the resulting behaviour is entirely up to the kernel to decide. So
> there's no guaranty that the optimization will last.
Exactly. I think the real question is whether you want to also give
the hint that the pages for that particular file should be first in
line to be discarded from the page cache.
> On the other hand, the whole point of posix_fadvise() is to give hints to
> the kernel so that he can decide on the best course of action. So I hope
> the interpretation above is one the main motivation behind that hint.
The main motivation is to make the pages easily discardable; the fact
that it happens to start writeback is really a side effect. So for
backup programs, including rsync when it is being used for backups,
using POSIX_FADV_DONTNEED is definitely a good idea. Whether or not
it is a good idea for dpkg really depends on whether you think the
files are going to be used soon after they are written --- either
because the user has just installed the new toy and wants to play with
it (i.e., "apt-get install tuxracer; tuxracer") or because of a
post-install script.
On the other hand, if the user was just updating a random set of
progams that aren't actually going to be used right away (i.e.,
"apt-get update; apt-get upgrade"), in that case the
POSIX_FADV_DONTNEED would probably be a good thing.
The reason why I suggested using sync_file_range() is because it is
very specifically directed at forcing the writeback to happen, which
is not quite the same thrust as posix_fadvise().
Regards,
- Ted
Reply to: