[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

[idea]: Switch default compression from "xz" to "zstd" for .deb packages


 Today I want to propose you to change default compression format in .deb,
 {data,control}.tar."xz" to ."zst".

 I want to hear your thought about this.

# Compared to past change to xz proposal (in DebConf12)

  There are reasons why I propose this

  * More bandwidth
  * More CPUs
  * More storage bandwidth

## More bandwidth

 According to https://www.speedtest.net/global-index, broadband bandwidth
 in Nicaragua becomes almost 10x

 - 2012: 1.7Mbps
 - 2023: 17.4Mbps

 10x faster than past: it means that file size is not so much problem for us

## More CPUs

 2012: ThinkPad L530 has Core i5-3320M (2 cores, 4 threads)
 2023: ThinkPad L15 has Core i5-1335U (10 cores, 12 threads)

  - i5-3320M: single 1614, multicore 2654
  - i5-1335U: single 3650, multicore 18076 points.

 And, xz cannot use multi core CPUs for decompression but zstd can.
 It means that if we use xz, we just get 2x CPU power, but change to zst,
 we can get more than 10x CPU power than 2012.

 It reduces a lot of time for package installation.

## More storage bandwidth

 SSD + PCIe 3/4/5 is enough, not be a blocker for decompression, now.

# Conclusion

  * More bandwidth
  * More CPUs
  * More storage bandwidth

  * Still limited: our resources (time!)

* So, if we change {data,control}.tar file format in .deb from xz to zst,
  we can reduce package installation time than ever since less decompression
  time. It saves our lifetime a bit :)

* Downside: package file size is a bit larger than now, but enough bandwidth
  will ease it for download time
  - Not sure how repository size will be, need an experiment

Hideki Yamane <henrich@iijmio-mail.jp>

Reply to: