Re: dpkg 1.15.6 is slow as hell
On Fri, 12 Mar 2010, Sven Joachim wrote:
> The decision to immediately fsync() all files written to disk has a
> detrimental influence on dpkg's performance, especially when unpacking
> large packages. On my system which has a 2.5" hard disk with an ext4
> filesystem, installing emacs23-common (containing 2123 files) with a hot
> cache takes 76 seconds, almost all of which is spent during unpacking.
> With dpkg 1.15.5.6, it takes 5.3 seconds, including processing three
> triggers. This is really painful. :-(
Not as bad here with ext3 but still worrying, taking gnome-icon-theme
(6534 files) I get ~3 seconds for 1.15.5.6 and 12 seconds for 1.15.6.
That's still a 300% increase.
Removing the single fsync() added in tarobject() completely restores the
original performance. Adding a single sync() after the whole unpack has
way less impact (1 or 2 seconds more).
diff --git a/src/archives.c b/src/archives.c
index ab10d19..69110c1 100644
--- a/src/archives.c
+++ b/src/archives.c
@@ -660,8 +660,6 @@ int tarobject(struct TarInfo *ti) {
am=(nifd->namenode->statoverride ? nifd->namenode->statoverride->mode : ti->Mode) & ~S_IFMT;
if (fchmod(fd,am))
ohshite(_("error setting permissions of `%.255s'"),ti->Name);
- if (fsync(fd))
- ohshite(_("unable to sync file '%.255s'"), ti->Name);
pop_cleanup(ehflag_normaltidy); /* fd= open(fnamenewvb.buf) */
if (close(fd))
ohshite(_("error closing/writing `%.255s'"),ti->Name);
diff --git a/src/processarc.c b/src/processarc.c
index 627bbd2..decf93d 100644
--- a/src/processarc.c
+++ b/src/processarc.c
@@ -637,6 +637,7 @@ void process_archive(const char *filename) {
close(p1[0]);
p1[0] = -1;
subproc_wait_check(c1, BACKEND " --fsys-tarfile", PROCPIPE);
+ sync();
if (oldversionstatus == stat_halfinstalled || oldversionstatus == stat_unpacked) {
/* Packages that were in `installed' and `postinstfailed' have been reduced
Other possibility would be to use the loop afterwards to reopen all
installed files and call fsync() on them. The disadvantage of sync() is
obviously when unrelated disk activity happens in parallel to dpkg, it
will have to wait more due to this.
Cheers,
--
Raphaël Hertzog
Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/
My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/
Reply to: