Bug#973861: apt: http acquire method still failing with "Undetermined error" or "Data left in the buffer"
On Thu, Jan 21, 2021 at 03:27:40PM +0000, Witold Baryluk wrote:
> Package: apt
> Version: 2.1.18
> Followup-For: Bug #973861
> X-Debbugs-Cc: witold.baryluk@gmail.com, mirror@init7.net
>
> I started getting this error recently too.
>
> To the point that using live-build (that downloads ~6100 package in one
> go) is not possible, because some package will fail to download and fail
> entire apt install request.
>
>
> This is related to keepalive.
>
> For example most nginx server has keepalive_requests 100 by default. That
> is, after serving 100 request, nginx server will close the connection.
> This is because some dynamic memory allocated by nginx is allocated on
> arenas that are tied to TCP connection, and they are never freed until
> the connection is closed. This is a high performance design, but to
> prevent leaks and abuse, the connection must be closed after some number
> of requests.
>
> Most tools (wget), respect the response from the server, close the
> existing connection and open a new one. It is easy to see with wget -i -
> and feeding the same file 10000 times. Every 100th request will start a
> new TCP connection.
>
>
> In apt this manifests as some XX00 request failing.
>
> Get:4500 http://mirror.init7.net/debian bullseye/main amd64 libplymouth5 amd64 0.9.5-2 [108 kB]
> Err:4500 http://mirror.init7.net/debian bullseye/main amd64 libplymouth5 amd64 0.9.5-2
> Undetermined Error [IP: 2001:1620::1620 80]
>
>
> Get:1600 http://mirror.init7.net/debian bullseye/main amd64 libastyle3 amd64 3.1-2+b1 [109 kB]
> Err:1600 http://mirror.init7.net/debian bullseye/main amd64 libastyle3 amd64 3.1-2+b1]
> Undetermined Error [IP: 2001:1620::1620 80]
>
>
>
> This is repeatable and reproducible. I can try 10 times doing the
> download, and every time the same file and number will fail. (Sometimes I
> might get lucky and one of them will succeed, but not ther other).
>
> I don't know why files 100, 200, 300, ... doesn't fail, but it looks like
> some bug in apt in general.
>
> (The above files do exist and are served correctly by the server).
>
>
> Does apt use maybe HTTP pipelining in addition to keepalive, that causes
> issues? Or maybe in general the retry logic is broken somehow.
Yes - pipelining is absolutely vital to get sensible performance.
Did you enable retries?
Anyway, go compile apt and add debugging code in appropriate places and
figure out where it's broken. I don't have a reproducer and nor the time
nor the ability to investigate where the breakage is - it generally
starts working before I've really had a chance to fix it.
>
> I wasn't able to reproduce this issue with Fastly CDN or Apache 2.4,
> probably because they use different keepalive request limit policy. It
> might be a bug in nginx too. I am not sure. The server I was using
> appears to be using nginx/1.14.2.
More likely a function of MTU or network speed or something than the
server software, and the concrete list of packages and the order in
which they are downloaded is important too
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
Reply to: