Re: [idea]: Switch default compression from "xz" to "zstd" for .deb packages
Hi!
On Sat, 2023-09-16 at 10:31:20 +0530, Hideki Yamane wrote:
> ## More bandwidth
>
> According to https://www.speedtest.net/global-index, broadband bandwidth
> in Nicaragua becomes almost 10x
>
> - 2012: 1.7Mbps
> - 2023: 17.4Mbps
Well that page still does not look too great for many other countries,
including with fixed broadband.
> ## More CPUs
>
> 2012: ThinkPad L530 has Core i5-3320M (2 cores, 4 threads)
> 2023: ThinkPad L15 has Core i5-1335U (10 cores, 12 threads)
>
> https://www.cpubenchmark.net/compare/817vs5294/Intel-i5-3320M-vs-Intel-i5-1335U
> - i5-3320M: single 1614, multicore 2654
> - i5-1335U: single 3650, multicore 18076 points.
>
> And, xz cannot use multi core CPUs for decompression but zstd can.
> It means that if we use xz, we just get 2x CPU power, but change to zst,
> we can get more than 10x CPU power than 2012.
That's not correct. dpkg-deb is doing multi-threaded xz decompression
since 1.21.13, and dpkg-source is doing multi-threaded xz compression
and decompression since 1.21.14.
Also the Ubuntu zstd implementation did not have multi-threaded support
at all, the one implemented in dpkg 1.21.18 does have explicit
multi-threaded support for _compression_, but AFIUI (from zstd code and
its API being used in libdpkg) not for decompression.
> It reduces a lot of time for package installation.
> * So, if we change {data,control}.tar file format in .deb from xz to zst,
> we can reduce package installation time than ever since less decompression
> time. It saves our lifetime a bit :)
>
> * Downside: package file size is a bit larger than now, but enough bandwidth
> will ease it for download time
> - Not sure how repository size will be, need an experiment
Thus these seem rather hand-wavy.
And in addition support for zstd at least in Debian seems rather poor:
https://wiki.debian.org/Teams/Dpkg/DebSupport
(Support in dpkg was mainly added to overcome the schism that Ubuntu
had created in the .deb format. :/)
Thanks,
Guillem
Reply to: