[RFC/PATCH 0/4] Re: Bug#605009: serious performance regression with ext4

[Jonathan Nieder <jrnieder@gmail.com>]

Hi Guillem,

Here are some rough patches implementing Ted's suggestions:

Ted Ts'o wrote:

>     extract(a);
>     sync_file_range(fd.a, 0, 0, SYNC_FILE_RANGE_WRITE); 
>     extract(b.dpkg-new);
>     sync_file_range(fd.b, 0, 0, SYNC_FILE_RANGE_WRITE); 
>     extract(c.dpkg-new);
>     sync_file_range(fd.c, 0, 0, SYNC_FILE_RANGE_WRITE); 
> 
>     sync_file_range(fd.a, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE); 
>     sync_file_range(fd.b, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE); 
>     sync_file_range(fd.c, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE); 
> 
>     fdatasync(a);
>     fdatasync(b.dpkg-new);
>     fdatasync(c.dpkg-new);
> 
>     rename(b.dpkg-new, b);
>     rename(c.dpkg-new, c);

Exceptions to what Ted did: this does not special-case new files and
it intersperses fdatasync and rename.  If that matters I'd be
interested to learn that.

Results (on ext4) suggest that patches 1 and 4 matter and the rest are
within noise.  Timings are rough; sometimes replicates vary by as much
as a second.  Numbers are cold cache (i.e., after running sync and
echo 3>.../drop_caches), best of 3, dpkg --install python2.7 and
python2.7-minimal.

before:
 5.73user 1.62system 0:33.84elapsed 21%CPU (0avgtext+0avgdata 89968maxresident)k
 0inputs+0outputs (0major+46962minor)pagefaults 0swaps

patch 1 (use SYNC_FILE_RANGE_WRITE):
 5.64user 1.69system 0:10.47elapsed 69%CPU (0avgtext+0avgdata 90000maxresident)k
 0inputs+0outputs (0major+46948minor)pagefaults 0swaps

patch 1+2 (use SYNC_FILE_RANGE_WAIT_BEFORE):
 5.48user 1.61system 0:10.43elapsed 70%CPU (0avgtext+0avgdata 90000maxresident)k
 0inputs+0outputs (0major+46958minor)pagefaults 0swaps

patch 1+2+3 (delay first fsync):
 5.63user 1.66system 0:10.45elapsed 70%CPU (0avgtext+0avgdata 89984maxresident)k
 0inputs+0outputs (0major+46966minor)pagefaults 0swaps

patch 1+2+3+4 (use fdatasync):
 5.65user 1.60system 0:10.24elapsed 71%CPU (0avgtext+0avgdata 90016maxresident)k
 0inputs+0outputs (0major+46977minor)pagefaults 0swaps

Jonathan Nieder (4):
  dpkg: On Linux initiate writeback of unpacked files ASAP
  dpkg: On Linux finish writeback before fsync
  dpkg: Write back all files before first fsync
  dpkg: Use fdatasync instead of fsync

 src/archives.c |   50 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 49 insertions(+), 1 deletions(-)