Re: Bug#588339: sync() causes significant dpkg unpack performance degradation on tmpfs (pbuilder, piuparts, ...)
Hi kernel team,
Andreas Beckmann wrote:
> Unfortunately since the last dpkg changes concerning sync()/fsync() this
> no longer works out well - the continuous sync() from a "virtual" chroot
> on tmpfs hits the physical system really hard, causing speed loss
> factors between 3-5, probably more if multiple pbuilder builds/piuparts
> tests are run in parallel.
Why would sync() do anything on tmpfs? The s_bdi field from its
superblock is never set to non-NULL in mm/shmem.c, so that’s not it.
Ah, but sync_filesystems() iterates over all filesystems, not just
those accessible from the chroot.
This sucks. To recap:
1. On ext4 with certain mount options, using rename() without first
calling fsync() to get the data on disk has an unfortunate risk of
clearing out a file[1].
2. On ext4 with certain mount options, using fsync() instead of sync()
to sync a collection of newly installed files is unacceptably
slow[2].
3. sync() obviously does way more than we want it too, since it
touches files and filesystems that have nothing to do with
dpkg’s work.
So what should we do? Dear kernel, we will happily provide a list
of files we want to be renamed in place. Can you make sure they
have the right data without _repeatedly_ incurring the penalty of
fsync()?
Jonathan
[1] http://bugs.debian.org/567089
[2] http://bugs.debian.org/578635
Reply to: