Re: dpkg with threaded xz decompression.
On 2022-12-15 05:50:34 [+0100], Guillem Jover wrote:
> Hi!
Hi,
> On Wed, 2022-12-14 at 07:15:40 +0100, Sebastian Andrzej Siewior wrote:
> > On 2022-10-07 05:07:08 [+0200], Guillem Jover wrote:
> > > Great, thanks much for this! I've rebased (with no changes needed) the
> > > patch on top of git HEAD, built and executed the functional test suite,
> > > and it seems to be working fine. :) And pushed it to
> > > <https://git.hadrons.org/git/debian/dpkg/dpkg.git/commit/?h=next/xz-mt-decompressor>.
> >
> > The 5.4 hit experimental as you noticed. Once you fine with your testing
> > we may move it to unstable.
>
> Yes, thank you! I'm running with it, and seems to be working fine. Did
> you end up doing more tests? I think moving the library to unstable the
I had the xz binary in use on a few systems and I had dpkg packaged to
use it. Nothing bad did happen.
> sooner the better, so that it can be exposed to more people. I guess I'd
> do a dpkg upload in at most a couple of days or so.
Okay. Let me do this today.
One side note: I had a bit of off-list discussion with Lasse regarding
the LZMA properties (lc/lp/pb) and the arm64 filter. It seems it
squeezes a few additional bytes during the compression. The arm64 filter
requires 5.4.0 but changing the properties is transparent. Do you think,
that we could tweak a little? Also he mentioned that the used memory
(for the dictionary and so the block) could be increased today since we
do have more memory. What do you think? Is this something we should
touch or is the benefit probably small because most packages are tiny
with a few exceptions (gcc, kernel, texlive,...) so it is not worh the
trouble with people that have just a little bit of ram.
> I think one minor concern I have (but now realized it does not affect
> Debian) is that the -T option for the xz tool (when used by libdpkg
> instead of linking against the liblzma library), will fallback to the
> non-multithreaded encoder/decoder if passed 1. But passing +1 is not
> supported by older versions. This would affect reproducibility. I
> guess I could always pass +1 and document that the new version is
> required when not linking against the library, but meh I guess.
Okay. Good point.
> I also realized that it would be nice to update the dpkg-source code
> to use the threaded stuff. Will prepare a patch for that.
Oh. I assumed it did it ;) Cool.
> Thanks,
> Guillem
Sebastian
Reply to: