[RFC/PATCH 0/4] Re: Bug#605009: serious performance regression with ext4
- To: debian-dpkg@lists.debian.org
- Cc: Ted Ts'o <tytso@mit.edu>, debian-kernel@lists.debian.org
- Subject: [RFC/PATCH 0/4] Re: Bug#605009: serious performance regression with ext4
- From: Jonathan Nieder <jrnieder@gmail.com>
- Date: Mon, 29 Nov 2010 00:48:25 -0600
- Message-id: <[🔎] 20101129064825.GA6750@burratino>
- In-reply-to: <20101129041152.GQ2767@thunk.org>
- References: <20101126093257.23480.86900.reportbug@pluto.milchstrasse.xx> <20101126145327.GB19399@rivendell.home.ouaza.com> <20101126215254.GJ2767@thunk.org> <20101127075831.GC24433@burratino> <20101127085346.GD14011@rivendell.home.ouaza.com> <20101129041152.GQ2767@thunk.org>
Hi Guillem,
Here are some rough patches implementing Ted's suggestions:
Ted Ts'o wrote:
> extract(a);
> sync_file_range(fd.a, 0, 0, SYNC_FILE_RANGE_WRITE);
> extract(b.dpkg-new);
> sync_file_range(fd.b, 0, 0, SYNC_FILE_RANGE_WRITE);
> extract(c.dpkg-new);
> sync_file_range(fd.c, 0, 0, SYNC_FILE_RANGE_WRITE);
>
> sync_file_range(fd.a, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE);
> sync_file_range(fd.b, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE);
> sync_file_range(fd.c, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE);
>
> fdatasync(a);
> fdatasync(b.dpkg-new);
> fdatasync(c.dpkg-new);
>
> rename(b.dpkg-new, b);
> rename(c.dpkg-new, c);
Exceptions to what Ted did: this does not special-case new files and
it intersperses fdatasync and rename. If that matters I'd be
interested to learn that.
Results (on ext4) suggest that patches 1 and 4 matter and the rest are
within noise. Timings are rough; sometimes replicates vary by as much
as a second. Numbers are cold cache (i.e., after running sync and
echo 3>.../drop_caches), best of 3, dpkg --install python2.7 and
python2.7-minimal.
before:
5.73user 1.62system 0:33.84elapsed 21%CPU (0avgtext+0avgdata 89968maxresident)k
0inputs+0outputs (0major+46962minor)pagefaults 0swaps
patch 1 (use SYNC_FILE_RANGE_WRITE):
5.64user 1.69system 0:10.47elapsed 69%CPU (0avgtext+0avgdata 90000maxresident)k
0inputs+0outputs (0major+46948minor)pagefaults 0swaps
patch 1+2 (use SYNC_FILE_RANGE_WAIT_BEFORE):
5.48user 1.61system 0:10.43elapsed 70%CPU (0avgtext+0avgdata 90000maxresident)k
0inputs+0outputs (0major+46958minor)pagefaults 0swaps
patch 1+2+3 (delay first fsync):
5.63user 1.66system 0:10.45elapsed 70%CPU (0avgtext+0avgdata 89984maxresident)k
0inputs+0outputs (0major+46966minor)pagefaults 0swaps
patch 1+2+3+4 (use fdatasync):
5.65user 1.60system 0:10.24elapsed 71%CPU (0avgtext+0avgdata 90016maxresident)k
0inputs+0outputs (0major+46977minor)pagefaults 0swaps
Jonathan Nieder (4):
dpkg: On Linux initiate writeback of unpacked files ASAP
dpkg: On Linux finish writeback before fsync
dpkg: Write back all files before first fsync
dpkg: Use fdatasync instead of fsync
src/archives.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 49 insertions(+), 1 deletions(-)
Reply to: