Re: Pre-approval request for dpkg sync() changes for squeeze
On 2010-11-15 08:21 +0100, Raphael Hertzog wrote:
> On Wed, 10 Nov 2010, Philipp Kern wrote:
>> On Tue, Nov 09, 2010 at 07:50:23PM +0100, Philipp Kern wrote:
>> > On Fri, Oct 22, 2010 at 11:35:54AM +0200, Guillem Jover wrote:
>> > > 1) Switch back from sync() to fsync() before rename() (while keeping
>> > > the sync() code around for the benefit of other distributions
>> > > that might not want to switch just yet). So to avoid unrelated
>> > > I/O when there's background work being done for example. This
>> > > hack also only works on Linux where sync() is synchronous, so
>> > > it would unify that code path for all dpkg supported platforms.
>> > >
>> > > Bug: #588339
>> > >
>> > > <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=87740373>
>> > I'm still not sure we have all needed information. Would you mind doing
>> > a matrix what this change would cause? I.e. if we get performance
>> > regressions on ext4, btrfs or even ext3 with the squeeze kernel (we're
>> > really only interested in that one, not some random other revision)
>> > or if we even get data safety regressions.
>>
>> I also forgot to throw nfs into the picture.
>
> I'm sorry, I won't have the time to do new benchmarks on this.
>
> The only benchmarks we have have been made by Sven Joachim:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578635#20
> (asyncsync is the switch to sync() instead of fsync() so the opposite of
> the above patch)
>
> He mentionned that without the sync() trick it takes 3 to 5 times longer
> to unpack a package.
Even longer actually, see the figures below.
> Sven, would you have time to provide some of the stats asked by the
> release team?
I can only test ext4, here are some samples of dpkg unpacking a large
package (dpkg --unpack --no-triggers emacs23-common_23.2+1-5.1_all.deb),
leaving out user and sys times since those do not vary much (~ 0.5
seconds in every case):
dpkg version Cache mount options unpack time
̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
1.15.8.5 cold defaults 7.803s
1.15.8.5 warm defaults 5.283s
1.15.8.5 cold nodelalloc 7.608s
1.15.8.5 warm nodelalloc 3.783s
1.15.7 cold defaults 40.429s
1.15.7 warm defaults 37.848s
1.15.7 cold nodelalloc 7.945s
1.15.7 warm nodelalloc 3.524s
All this is with a standard squeeze kernel on an otherwise idle system.
It should be noted that with lots of other disk activity such as writing
to USB disks, the figures in dpkg 1.15.8.5 can become much worse and
dpkg might even stall because of the many sync() calls:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595927.
As far as ext4 is concerned, switching back to fsync() seems to be
acceptable only if the filesystem is mounted with the nodelalloc
option. Maybe the installer should set this up.
Regards,
Sven
Reply to: