Re: Switching default dpkg-deb compressor to xz
On Wed, May 08, 2013 at 06:14:36PM +0200, Bastian Blank wrote:
> On Tue, May 07, 2013 at 09:49:03PM +0200, Guillem Jover wrote:
> > As mentioned some months ago [0], I'm planning to switch dpkg-deb default
> > compressor from gzip to xz, as there seemed to be consensus that was
> > the way to go, and given the amount of already manually switched
> > packages, or packaging helpers. :/
>
> What about the compression level? xz -6 is pretty heavy and not needed
> for 99% of the packages. -3 or even -2 or -1 are sufficient.
As my and Hideki's repacks of the archive show, special-casing small
packages is a waste of time: gains are hardly below linear for any
packages big enough to take longer than fork()ing the compressor.
Quoting some data from 2011, all with xz -6:
] * A repack of the whole archive (amd64+all main, ~40GB) took close to three
] hours on a 6xPhenomII 2.8GHz box (ar p|gzip/bzip2 -d|xz).
]
] Does someone have an estimate how many core-hours would an archive rebuild
] on such a machine take? Folks on IRC quoted numbers like "340", "240 on a
] very fast box", "more like 1500" -- too divergent for my liking. The
] first number, 340, would mean switching to xz exclusively would increase
] average build time by ~5%.
]
] * armel Cortex-A8 600MHz does xz consistently 12.1 times slower than one
] core of the above box (on a big compressible and a big uncompressible
] file), that's ~2.6 times slower per-MHz.
]
] Glancing at build logs of a few randomly chosen packages, I see armel
] builds taking respectively 16.9, 13.1, 18.8 and 15.1 times longer. Not
] sure what are the actual speeds of buildds, but it looks like armel would
] be affected by less than the above 5%.
I'd thus suggest using the default, -6, everywhere other than perhaps
openclipart (already compressed) and the likes. xz folks chose this value
for a reason :)
--
ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ
Reply to: