[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Switching default dpkg-deb compressor to xz

On Wed, May 08, 2013 at 06:14:36PM +0200, Bastian Blank wrote:
> On Tue, May 07, 2013 at 09:49:03PM +0200, Guillem Jover wrote:
> > As mentioned some months ago [0], I'm planning to switch dpkg-deb default
> > compressor from gzip to xz, as there seemed to be consensus that was
> > the way to go, and given the amount of already manually switched
> > packages, or packaging helpers. :/
> What about the compression level? xz -6 is pretty heavy and not needed
> for 99% of the packages. -3 or even -2 or -1 are sufficient.

As my and Hideki's repacks of the archive show, special-casing small
packages is a waste of time: gains are hardly below linear for any
packages big enough to take longer than fork()ing the compressor.

Quoting some data from 2011, all with xz -6:
] * A repack of the whole archive (amd64+all main, ~40GB) took close to three
]   hours on a 6xPhenomII 2.8GHz box (ar p|gzip/bzip2 -d|xz).
]   Does someone have an estimate how many core-hours would an archive rebuild
]   on such a machine take?  Folks on IRC quoted numbers like "340", "240 on a
]   very fast box", "more like 1500" -- too divergent for my liking.  The
]   first number, 340, would mean switching to xz exclusively would increase
]   average build time by ~5%.
] * armel Cortex-A8 600MHz does xz consistently 12.1 times slower than one
]   core of the above box (on a big compressible and a big uncompressible
]   file), that's ~2.6 times slower per-MHz.
]   Glancing at build logs of a few randomly chosen packages, I see armel
]   builds taking respectively 16.9, 13.1, 18.8 and 15.1 times longer.  Not
]   sure what are the actual speeds of buildds, but it looks like armel would
]   be affected by less than the above 5%.

I'd thus suggest using the default, -6, everywhere other than perhaps
openclipart (already compressed) and the likes.  xz folks chose this value
for a reason :)


Reply to: