[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [idea]: Switch default compression from "xz" to "zstd" for .deb packages


On Sat, 2023-09-16 at 10:31:20 +0530, Hideki Yamane wrote:
> ## More bandwidth
>  According to https://www.speedtest.net/global-index, broadband bandwidth
>  in Nicaragua becomes almost 10x
>  - 2012: 1.7Mbps
>  - 2023: 17.4Mbps

Well that page still does not look too great for many other countries,
including with fixed broadband.

> ## More CPUs
>  2012: ThinkPad L530 has Core i5-3320M (2 cores, 4 threads)
>  2023: ThinkPad L15 has Core i5-1335U (10 cores, 12 threads)
>  https://www.cpubenchmark.net/compare/817vs5294/Intel-i5-3320M-vs-Intel-i5-1335U
>   - i5-3320M: single 1614, multicore 2654
>   - i5-1335U: single 3650, multicore 18076 points.
>  And, xz cannot use multi core CPUs for decompression but zstd can.
>  It means that if we use xz, we just get 2x CPU power, but change to zst,
>  we can get more than 10x CPU power than 2012.

That's not correct. dpkg-deb is doing multi-threaded xz decompression
since 1.21.13, and dpkg-source is doing multi-threaded xz compression
and decompression since 1.21.14.

Also the Ubuntu zstd implementation did not have multi-threaded support
at all, the one implemented in dpkg 1.21.18 does have explicit
multi-threaded support for _compression_, but AFIUI (from zstd code and
its API being used in libdpkg) not for decompression.

>  It reduces a lot of time for package installation.

> * So, if we change {data,control}.tar file format in .deb from xz to zst,
>   we can reduce package installation time than ever since less decompression
>   time. It saves our lifetime a bit :)
> * Downside: package file size is a bit larger than now, but enough bandwidth
>   will ease it for download time
>   - Not sure how repository size will be, need an experiment

Thus these seem rather hand-wavy.

And in addition support for zstd at least in Debian seems rather poor:


(Support in dpkg was mainly added to overcome the schism that Ubuntu
had created in the .deb format. :/)


Reply to: