Re: .deb format: let's use 0.939, zstd, drop bzip2
On Wed, May 08, 2019 at 09:53:19PM +0100, Ian Jackson wrote:
> (adding debian-dpkg)
Probably too late to move the thread :(
> Adam Borowski writes (".deb format: let's use 0.939, zstd, drop bzip2"):
> > First, the 0.939 format, as described in "man deb-old". While still being
> > accepted by dpkg, it had been superseded before even the very first stable
> > release. Why? It has at least two upsides over 2.0:
> > * it's faster by a small but non-negligible factor. A task "unpack all
> > packages in default XFCE GUI install" gets done by stock dpkg (after
> > repacking everything as gzip) 3% faster.
>
> I'm not sure why it should be faster.
Now this is interesting; retested in a reproducible way:
Repack tools: http://ix.io/1HNB http://ix.io/1HNC
Corpus used: http://ix.io/1Iyg
Benchmark harness (preceded by warm-up to avoid thermal effects):
for x in gz0.93 gz2;do for i in {0..100};do rm -rf /tmp/target;/usr/bin/time -p -o /dev/stdout parallel -i dpkg-deb -x '{}' /tmp/target -- $x/*.deb;done|tee /tmp/$x;done
Results (median):
gz0.93: real 1.28 user 19.68
gz2: real 1.33 user 20.67
But, with -j1, I got real 16.14 for both in the first run (median of 11).
> As the person who deprecated deb-old in favour of the current format,
> my motives were:
> - the old format was a real pain to unpack without a custom utility
> (this used to be a much more serious problem)
I understand that this was an issue when prototyping, but have you used ar
to manipulate .deb archives even once this millenium? By now, the deb
format is common, while ar is an obscure implementation detail.
> - the old format was not very extensible.
As in, can't change the compression used?
Only after looking at ways to add 0.93:zstd to real dpkg I noticed that it
relies on 2.0 ar's member filenames to select decompressor. In my own
experiments, I used libarchive which doesn't look at file extensions at all.
> We use the extensibility for compression format changes, but
> compressors all have magic numbers and we could just use those.
Right.
> It would be much less convenient to change our archive format from tar
> to something else, as proposed by Ansgar, without this extensibility.
> (I don't necessarily think Ansgar's idea is a good one, but it makes
> an example here.)
The control tarball allows a lot of room for extensions, and it has been
used for them many times.
Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ Have you accepted Khorne as your lord and saviour?
⠈⠳⣄⠀⠀⠀⠀
Reply to: