Re: Speeding up dpkg, a proposal
ext Raphael Hertzog <hertzog@debian.org> writes:
> On Wed, 02 Mar 2011, Marius Vollmer wrote:
>> - Instead, we move all packages that are to be unpacked into
>> half-installed / reinstreq before touching the first one, and put a
>> big sync() right before carefully writing /var/lib/dpkg/status.
>
> The big sync() doesn't work. It means dpkg never finishes its work on
> systems with lots of unrelated I/O.
Ok, understood. It's now clear to me that the big sync should be
replaced with deferred fsyncs. (I would defer the fsync of the content
of all packages until modstatdb_checkpoint, not just until
tar_deferred_extract.)
With that change, do you think the approach is sound?
> We've seen reports of poor performance with btrfs (and that's what you use
> for Meego IIRC) so you might want to investigate why btrfs is coping so
> badly with a few fsync() just on the status files.
This is about Harmattan, which uses ext4.
To understand our troubles, you need to know that we have around 2500
packages with just a single file in it. For those packages, dpkg spends
the largest part of its time in writing the nine journal entries to
/var/lib/dpkg/updates.
We will reduce the number of our packages, so this issue might solve
itself that way, but I had good success in reducing the per-package
overhead of dpkg, and if it is correct and works for us, why not use the
'reckless' option as well?
Reply to: