Bug#978671: marked as done (apt http transport gets stuck after package download)
Your message dated Tue, 29 Dec 2020 23:33:06 +0100
with message-id <20201229232653.GA1389454@debian.org>
and subject line Re: Bug#978671: apt http transport gets stuck after package download
has caused the Debian Bug report #978671,
regarding apt http transport gets stuck after package download
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)
--
978671: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978671
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message ---
Package: apt
Version: 1.8.2.1
Severity: normal
Tags: patch
Dear Maintainer,
On Debian Buster we're seeing 30s timeouts when downloading
a particular set of debian packages from an internal repo:
```
$ time apt-get download -y rubygem-stud rubygem-mustache rubygem-insist rubygem-pleaserun rubygem-fpm
...
Fetched 451 kB in 30s (15.0 kB/s)
real 0m31.022s
user 0m0.923s
sys 0m0.080s
```
The issue is fixed in the following commit in apt 2.1.9:
```
commit 08f05aa8beb58fa32485e2087eb21a9f3cb267bb
Author: Julian Andres Klode <julian.klode@canonical.com>
Date: Mon Jun 29 14:03:21 2020 +0200
http: Redesign reading of pending data
Instead of reading the data early, disable the timeout for the
select() call and read the data later. Also, change Read() to
call only once to drain the buffer in such instances.
We could optimize this to call read() multiple times if there
is also pending stuff on the socket, but that it slightly more
complex and should not provide any benefits.
```
Please consider backporting this into Debian Buster,
as it fixes the problem and the patch itself applies cleanly.
-- System Information:
Debian Release: 10.6
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.4.70
Kernel taint flags: TAINT_OOT_MODULE
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE=C (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: unable to detect
Versions of packages apt depends on:
ii adduser 3.118
ii debian-archive-keyring 2019.1
ii gpgv 2.2.12-1+deb10u1
ii libapt-pkg5.0 1.8.2.1
ii libc6 2.28-10
ii libgcc1 1:8.3.0-6
ii libgnutls30 3.6.7-4+deb10u5
ii libseccomp2 2.5.1-1
ii libstdc++6 8.3.0-6
Versions of packages apt recommends:
ii ca-certificates 20200601~deb10u1
Versions of packages apt suggests:
pn apt-doc <none>
pn aptitude | synaptic | wajig <none>
ii dpkg-dev 1.19.7
ii gnupg 2.2.12-1+deb10u1
ii gnupg2 2.2.12-1+deb10u1
pn powermgmt-base <none>
-- no debconf information
--- End Message ---
--- Begin Message ---
On Tue, Dec 29, 2020 at 09:25:12PM +0000, Ivan Babrou wrote:
> Package: apt
> Version: 1.8.2.1
> Severity: normal
> Tags: patch
>
> Dear Maintainer,
>
> On Debian Buster we're seeing 30s timeouts when downloading
> a particular set of debian packages from an internal repo:
>
> ```
> $ time apt-get download -y rubygem-stud rubygem-mustache rubygem-insist rubygem-pleaserun rubygem-fpm
> ...
> Fetched 451 kB in 30s (15.0 kB/s)
>
> real 0m31.022s
> user 0m0.923s
> sys 0m0.080s
> ```
>
> The issue is fixed in the following commit in apt 2.1.9:
>
> ```
> commit 08f05aa8beb58fa32485e2087eb21a9f3cb267bb
> Author: Julian Andres Klode <julian.klode@canonical.com>
> Date: Mon Jun 29 14:03:21 2020 +0200
>
> http: Redesign reading of pending data
>
> Instead of reading the data early, disable the timeout for the
> select() call and read the data later. Also, change Read() to
> call only once to drain the buffer in such instances.
>
> We could optimize this to call read() multiple times if there
> is also pending stuff on the socket, but that it slightly more
> complex and should not provide any benefits.
> ```
>
> Please consider backporting this into Debian Buster,
> as it fixes the problem and the patch itself applies cleanly.
I'm sorry to disappoint you, but we cannot risk backporting these
changes to stable. This code is highly fragile and it seems we now got
it stable again, but backporting it to stable is a no-go.
We have no idea how all those commits interact with each other and
seemingly "obviously correct" commits caused massive regressions the
last time we tried this. I'm happy it seems to work now.
The only thing we can do is backport the entire http code as-is, but
cherry-picking individual changes has been proven to be a recipe for
disaster. That however I do not believe is a suitable change for stable.
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
--- End Message ---
Reply to: