[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Pre-approval request for dpkg sync() changes for squeeze

On 2010-11-15 08:21 +0100, Raphael Hertzog wrote:

> On Wed, 10 Nov 2010, Philipp Kern wrote:
>> On Tue, Nov 09, 2010 at 07:50:23PM +0100, Philipp Kern wrote:
>> > On Fri, Oct 22, 2010 at 11:35:54AM +0200, Guillem Jover wrote:
>> > >   1) Switch back from sync() to fsync() before rename() (while keeping
>> > >      the sync() code around for the benefit of other distributions
>> > >      that might not want to switch just yet). So to avoid unrelated
>> > >      I/O when there's background work being done for example. This
>> > >      hack also only works on Linux where sync() is synchronous, so
>> > >      it would unify that code path for all dpkg supported platforms.
>> > > 
>> > >      Bug: #588339
>> > > 
>> > >   <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=87740373>
>> > I'm still not sure we have all needed information.  Would you mind doing
>> > a matrix what this change would cause?  I.e. if we get performance
>> > regressions on ext4, btrfs or even ext3 with the squeeze kernel (we're
>> > really only interested in that one, not some random other revision)
>> > or if we even get data safety regressions.
>> I also forgot to throw nfs into the picture.
> I'm sorry, I won't have the time to do new benchmarks on this. 
> The only benchmarks we have have been made by Sven Joachim:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578635#20
> (asyncsync is the switch to sync() instead of fsync() so the opposite of
> the above patch)
> He mentionned that without the sync() trick it takes 3 to 5 times longer
> to unpack a package.

Even longer actually, see the figures below.

> Sven, would you have time to provide some of the stats asked by the
> release team?

I can only test ext4, here are some samples of dpkg unpacking a large
package (dpkg --unpack --no-triggers emacs23-common_23.2+1-5.1_all.deb),
leaving out user and sys times since those do not vary much (~ 0.5
seconds in every case):

dpkg version	Cache	mount options  unpack time    
̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅        cold    defaults        7.803s        warm    defaults        5.283s        cold    nodelalloc      7.608s        warm    nodelalloc      3.783s 
1.15.7          cold    defaults       40.429s 
1.15.7          warm    defaults       37.848s 
1.15.7          cold    nodelalloc      7.945s 
1.15.7          warm    nodelalloc      3.524s

All this is with a standard squeeze kernel on an otherwise idle system.
It should be noted that with lots of other disk activity such as writing
to USB disks, the figures in dpkg can become much worse and
dpkg might even stall because of the many sync() calls:

As far as ext4 is concerned, switching back to fsync() seems to be
acceptable only if the filesystem is mounted with the nodelalloc
option.  Maybe the installer should set this up.


Reply to: