[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Speeding up dpkg, a proposal

ext Raphael Hertzog <hertzog@debian.org> writes:

> On Wed, 02 Mar 2011, Marius Vollmer wrote:
>> - Instead, we move all packages that are to be unpacked into
>>   half-installed / reinstreq before touching the first one, and put a
>>   big sync() right before carefully writing /var/lib/dpkg/status.
> The big sync() doesn't work. It means dpkg never finishes its work on
> systems with lots of unrelated I/O.

Ok, understood.  It's now clear to me that the big sync should be
replaced with deferred fsyncs.  (I would defer the fsync of the content
of all packages until modstatdb_checkpoint, not just until

With that change, do you think the approach is sound?

> We've seen reports of poor performance with btrfs (and that's what you use
> for Meego IIRC) so you might want to investigate why btrfs is coping so
> badly with a few fsync() just on the status files.

This is about Harmattan, which uses ext4.

To understand our troubles, you need to know that we have around 2500
packages with just a single file in it.  For those packages, dpkg spends
the largest part of its time in writing the nine journal entries to

We will reduce the number of our packages, so this issue might solve
itself that way, but I had good success in reducing the per-package
overhead of dpkg, and if it is correct and works for us, why not use the
'reckless' option as well?

Reply to: