[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Pre-approval request for dpkg sync() changes for squeeze



Hi!

On Mon, 2010-11-15 at 08:21:05 +0100, Raphael Hertzog wrote:
> On Wed, 10 Nov 2010, Philipp Kern wrote:
> > On Tue, Nov 09, 2010 at 07:50:23PM +0100, Philipp Kern wrote:
> > > On Fri, Oct 22, 2010 at 11:35:54AM +0200, Guillem Jover wrote:
> > > >   1) Switch back from sync() to fsync() before rename() (while keeping
> > > >      the sync() code around for the benefit of other distributions
> > > >      that might not want to switch just yet). So to avoid unrelated
> > > >      I/O when there's background work being done for example. This
> > > >      hack also only works on Linux where sync() is synchronous, so
> > > >      it would unify that code path for all dpkg supported platforms.
> > > > 
> > > >      Bug: #588339
> > > > 
> > > >   <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=87740373>
> > > I'm still not sure we have all needed information.  Would you mind doing
> > > a matrix what this change would cause?  I.e. if we get performance
> > > regressions on ext4, btrfs or even ext3 with the squeeze kernel (we're
> > > really only interested in that one, not some random other revision)
> > > or if we even get data safety regressions.
> > 
> > I also forgot to throw nfs into the picture.

I think I provided all relevant information in this thread. Anyway a
short summary regarding performance would be:

 * ext3 does not have significant difference with either sync() or
   fsync().
 * ext4 performs slower with fsync() + mount defaults than sync() +
   mount defaults, should do as ext3 with either fsync() or sync()
   with nodelalloc.
 * btrfs should perform slower using sync() than fsync() (as I understood
   from Modestas Vainius and Mike Hommey comments, I've not seen actual
   benchmark results checking each against the other though).

And regarding data safety:

 * ext3 should not lose data even w/o sync() or fsync(), at most the
   status db will get out of sync relative to what's extracted on disk.
 * ext4 *will* get zero-length files w/o sync() or fsync() in dpkg, but
   *will* also get those with any other software not doing the syncs
   (i.e. most stuff in Debian) with default mount options anyway. So
   ext4 by default is not safe against data loss. nodelalloc should
   supposedly fix that.
 * btrfs, I don't really know, but I don't expect it to have
   zero-length file issues? In that case similar to ext3.

> I'm sorry, I won't have the time to do new benchmarks on this. 

Me neither.

> The only benchmarks we have have been made by Sven Joachim:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578635#20
> (asyncsync is the switch to sync() instead of fsync() so the opposite of
> the above patch)

Jonathan Nieder also provided some benchmarks in that same bug report
<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578635#25>. And
Modestas Vainius on
<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=588254>.
Although against slightly later kernel versions, but I don't think
that changes much.

> We really need to upload dpkg this week so we must take a decision what
> goes in.

Well, given #595927/#600075, there does not seem much more we can do
but apply those two patches, so if there's not compelling argument
put forward I'll just be applying them for next release targetting
squeeze.

thanks,
guillem


Reply to: