multi threaded support for xz
xz-utils 5.2.2 with threading support for the compressor is currently
in the deferred queue for another 24 hours [0].
Once this version has been built a binNMU of dpkg will pick up the
threading support. dpkg will the use the number of online CPUs for
compression [1] in a "dpkg-deb -b" invocation. Using more CPUs here
increases the required amount of memory. If the buildds start running
out of memory during dpkg-deb or start swapping - this might due to this
change.
There is lzma_stream_encoder_mt_memusage() which could be used to
compute the needed memory upfront and then maybe decrease the number of
selected CPUs while the memory limit is exceeded [2]. Also
dpkg-buildpackage's -j argument could be used as the initial hint
instead of number of online CPUs.
Just some thoughts in case something goes wrong :)
[0] https://ftp-master.debian.org/deferred.html
[1] https://sources.debian.net/src/dpkg/1.18.10/lib/dpkg/compress.c/#L534
[2] https://git.breakpoint.cc/cgit/bigeasy/xz-utils-debian.git/tree/src/xz/coder.c#n273
Sebastian
Reply to: