Thanks for the quick reply.
On Mon, Jan 7, 2019 at 7:37 PM Julian Andres Klode <
jak@debian.org> wrote:
On Mon, Jan 07, 2019 at 06:40:02PM +0100, Michael Stapelberg wrote:
> Package: apt
> Version: 1.8.0~alpha3
> Severity: normal
>
> For the past few weeks, I haven’t been able to update my Debian system. I
> finally took some time to look into what’s going wrong.
We'd have to figure out when this started, I guess.
Sorry, didn’t keep track of that.
>
> The symptom is that “apt update” just hung forever after fetching all files from
> my configured mirrors.
>
> A closer look revealed 100% CPU usage of 1 core in the
> /usr/lib/apt/methods/store process, which was writing to files in
> /var/lib/apt/lists/partial.
>
> Attaching gdb revealed that store was spending its time in a zstd function:
>
> #0 0x00007f376cc0bf78 in ?? () from /usr/lib/x86_64-linux-gnu/libzstd.so.1
> #1 0x00007f376cbcc275 in ?? () from /usr/lib/x86_64-linux-gnu/libzstd.so.1
> #2 0x00007f376cbcdedf in ?? () from /usr/lib/x86_64-linux-gnu/libzstd.so.1
> #3 0x00007f376cbce461 in ZSTD_compressContinue () from /usr/lib/x86_64-linux-gnu/libzstd.so.1
> #4 0x00007f376cbcfe9e in ?? () from /usr/lib/x86_64-linux-gnu/libzstd.so.1
> #5 0x00007f376d9874af in ?? () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
> #6 0x00007f376d97ffd1 in FileFd::Write(void const*, unsigned long long) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
> #7 0x000055afaedf1073 in StoreMethod::Fetch(pkgAcqMethod::FetchItem*) () at include/apt-pkg/fileutl.h:168
> #8 0x00007f376d939f83 in pkgAcqMethod::Run(bool) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
> #9 0x000055afaedf0759 in main () at /usr/include/c++/8/bits/basic_string.h:252
> #10 0x00007f376d32e09b in __libc_start_main (main=0x55afaedf06e0 <main>, argc=1, argv=0x7ffe0782f238, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe0782f228) at ../csu/libc-start.c:308
> #11 0x000055afaedf083a in _start () at ../methods/store.cc:147
>
> Sure enough, raising the cost of the zstd compression in my apt config fixed the
> issue:
>
> % cat /etc/apt/apt.conf.d/99disable-zstd
> APT::Compressor::zstd::Cost "999";
>
> I have a few remarks:
>
> 1. Why are the lists re-compressed at all?
> (Also note that I set Acquire::GzipIndexes "yes", so I have already signaled
> that I would prefer to accept the lists as they are.)
No. You have indicated that you want them stored compressed. Which picks the
cheapest compressor, that's usually lz4.
Ugh. Is there a way to tell apt to not bother re-compressing files ever?
Yes, it used to mean to not uncompress files and take them directly, but that
changed a few years ago, in 1.2 or so, as lz4 basically has not much overhead,
so it's much faster than keeping xz indexes around.
>
> 2. Why is store only using 1 of 16 CPU cores for compression?
store is using 1 process per file (up to a limit that's a multiple of your
cores), parallel compression of single files is meh.
That’s not what I’m observing. For me, precisely 1 store process is running, and it seems to be handling all files sequentially.
>
> 3. Should we disable zstd for the time being, given that it turns a 1-minute apt
> update into a many-minute apt update, to the point where I had assumed it was
> hanging?
It's not used by default. It's unclear to me why this happens to you. Do you have
another config file which sets a lower zstd cost perhaps?
No, it doesn’t look like it:
% grep -ri zst /etc/apt
/etc/apt/apt.conf.d/99disable-zstd:APT::Compressor::zstd::Cost "999";
%
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en