[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#810796: HTTP pipelining is broken and causes download failures



Package: apt
Version: 1.10
User: ubuntu-devel@lists.ubuntu.com
Usertags: origin-ubuntu xenial

apt enables pipelining by default. But it does not check to see if the
server supports HTTP/1.1, so where the server doesn't apt is breaking
the HTTP protocol by attempting to use pipelining. This causes download
failures.

Workaround: "Acquire::http::Pipeline-Depth 0;" makes it work. It might
be an idea to drop the pipelining default until this is fixed.

In my case, I have http://ftp.debian.org/debian in my sources.list, but
go through a squid 2.7.STABLE7 proxy as the machine running apt doesn't
have direct access to the Internet. This is done explicitly using a
Acquire::HTTP::Proxy setting (so not using transparent interception).

I'm getting "Hash Sum mismatch" errors on some downloaded debs. Right
now I get this failure if I have a sid chroot created by "mk-sbuild"
(from ubuntu-dev-tools) and attempt to "apt-get build-dep mysql-5.6"
through my proxy. I'm not sure how reproducible this might be for
others, but I can reproduce it reliably right now. Unfortunately
behaviour might also change as sid changes.

I captured apt's traffic to my proxy. and in the failure case I see it
requesting pipelined files immediately, which I think is wrong of apt,
followed by HTTP/1.0 "Connection: close" responses from the proxy (which
is why I think it is wrong of apt, since it should only request
pipelining with HTTP/1.1 servers).

In most cases the server returns the first file only and then apt
(perhaps in response to the Connection: close?) closes the connection
gracefully. Then apt makes other pipelined requests for the files it did
not successfully retrieve because they were not the first in its
invalid pipelined requests.

In some cases I see the server hasn't finished sending the first file
before I see an RST. In this case apt doesn't get the whole file,
doesn't retry it and this leads to a Hash Sum mismatch.

Speculation: the server always tries to send an RST after processing the
first request if it sees further incoming data at this time, since that
is in violation of HTTP/1.0. apt closes the connection gracefully after
it sees either an HTTP/1.0 response or a "Connection: close" (not sure
which, or maybe both). So this is a race. If the RST arrives and is
processed first, then apt gets partial data and fails. But the race only
exists because apt invalidly attempts to pipeline when it shouldn't.

Thanks,

Robie

Attachment: signature.asc
Description: Digital signature


Reply to: