Bug#64530: marked as done (apt: The http method does not work correctly for HTTP/1.0 proxies)
Your message dated Tue, 23 May 2000 10:42:37 -0600 (MDT)
with message-id <[🔎] Pine.LNX.3.96.1000523103048.3868C-100000@wakko.deltatee.com>
and subject line Bug#64530: apt: The http method does not work correctly for HTTP/1.0 proxies
has caused the attached Bug report to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what I am
talking about this indicates a serious mail system misconfiguration
somewhere. Please contact me immediately.)
Darren Benham
(administrator, Debian Bugs database)
--------------------------------------
Received: (at submit) by bugs.debian.org; 23 May 2000 05:46:53 +0000
>From amb@gedanken.demon.co.uk Tue May 23 00:46:53 2000
Return-path: <amb@gedanken.demon.co.uk>
Received: from anchor-post-34.mail.demon.net [194.217.242.92]
by master.debian.org with esmtp (Exim 3.12 #2 (Debian))
id 12u7Wm-0002ll-00; Tue, 23 May 2000 00:46:52 -0500
Received: from gedanken.demon.co.uk ([158.152.211.20] helo=g2.gedanken)
by anchor-post-34.mail.demon.net with esmtp (Exim 2.12 #1)
id 12u7Wj-00090P-0Y
for submit@bugs.debian.org; Tue, 23 May 2000 06:46:50 +0100
Received: from amb by g2.gedanken with local (Exim 3.12 #1 (Debian))
id 12txUf-0000JV-00; Mon, 22 May 2000 20:04:01 +0100
From: amb@gedanken.demon.co.uk
Subject: apt: The http method does not work correctly for HTTP/1.0 proxies
To: submit@bugs.debian.org
X-Mailer: bug 3.2.10
Message-Id: <[🔎] E12txUf-0000JV-00@g2.gedanken>
Date: Mon, 22 May 2000 20:04:01 +0100
Delivered-To: submit@bugs.debian.org
Package: apt
Version: 0.3.18
Severity: normal
The current version of the apt http method has some support for HTTP/1.0 servers
and proxies, but is not consistent.
-------------------- HttpMethod::SendReq() --------------------
/* Build the request. We include a keep-alive header only for non-proxy
requests. This is to tweak old http/1.0 servers that do support keep-alive
but not HTTP/1.1 automatic keep-alive. Doing this with a proxy server
will glitch HTTP/1.0 proxies because they do not filter it out and
pass it on, HTTP/1.1 says the connection should default to keep alive
and we expect the proxy to do this */
-------------------- HttpMethod::SendReq() --------------------
This is good, it shows that the workings of HTTP/1.0 and HTTP/1.1 are recognised
and that HTTP/1.0 will close the connection.
-------------------- ServerState::HeaderLine() --------------------
if (stringcasecmp(Tag.begin(),Tag.begin()+4,"HTTP") == 0)
{
// Evil servers return no version
if (Line[4] == '/')
{
if (sscanf(Line.c_str(),"HTTP/%u.%u %u %[^\n]",&Major,&Minor,
&Result,Code) != 4)
return _error->Error("The http server sent an invalid reply header");
}
else
{
Major = 0;
Minor = 9;
if (sscanf(Line.c_str(),"HTTP %u %[^\n]",&Result,Code) != 2)
return _error->Error("The http server sent an invalid reply header");
}
return true;
}
-------------------- ServerState::HeaderLine() --------------------
This is the code that checks for the returned header from the server. There is
no checking here if the server is HTTP/1.0 or HTTP/1.1. I think that you need
the following two lines before the return statement.
if(Major==0 || (Major==1 && Minor==0))
Encoding = Closes;
-------------------- ServerState::HeaderLine() --------------------
if (stringcasecmp(Tag,"Content-Length:") == 0)
{
if (Encoding == Closes)
Encoding = Stream;
-------------------- ServerState::HeaderLine() --------------------
Here the if statement says that if there is a "Content-Length:" header then the
connection will not close. This is different to what is said in the comment in
HttpMethod::SendReq(), there is no reason for an HTTP/1.0 server to not send
this header. You need to remove those two lines.
-------------------- ServerState::HeaderLine() --------------------
-------------------- ServerState::HeaderLine() --------------------
There is no code in this function to test for "Connection:" headers like
HTTP/1.1 says there might be. A "Connection: close" from an HTTP/1.1 server
means that the connection will be closed.
I think that you need code like this:
if (stringcasecmp(Tag,"Connection:") == 0)
{
if (stringcasecmp(Val,"Close") == 0)
Encoding = Closes;
}
Having tried these modifications on an HTTP/1.0 proxy I have noticed a much
better performance. Previously about half of the requests to the server were
never made (checking the server logs). I think this is because they were made
to the same connection as the previous request. With these changes all except
one requests work. I am still trying to debug that one since the correct amount
of data seems to have been sent, but apt reports that gzip cannot unzip it.
-- System Information
Debian Release: 2.2
Kernel Version: Linux g2 2.2.14 #1 Sun Apr 16 09:42:29 BST 2000 i586 unknown
Versions of the packages apt depends on:
ii libc6 2.1.3-10 GNU C Library: Shared libraries and Timezone
ii libstdc++2.10 2.95.2-10 The GNU stdc++ library
---------------------------------------
Received: (at 64530-done) by bugs.debian.org; 23 May 2000 16:42:32 +0000
>From jgg@ualberta.ca Tue May 23 11:42:32 2000
Return-path: <jgg@ualberta.ca>
Received: from hawkmoth.tac.net (telusvelocity.net) [205.233.109.54]
by master.debian.org with smtp (Exim 3.12 #2 (Debian))
id 12uHlI-0007Vp-00; Tue, 23 May 2000 11:42:32 -0500
Received: (qmail 10008 invoked from network); 23 May 2000 16:42:26 -0000
Received: from unknown (HELO wakko.deltatee.com) (209.115.196.25)
by hawkmoth.tac.net with SMTP; 23 May 2000 16:42:26 -0000
Received: from localhost (wakko.deltatee.com) [127.0.0.1] (jgg)
by wakko.deltatee.com with smtp (Exim 2.11 #1)
id 12uHlN-000128-00 (Debian); Tue, 23 May 2000 10:42:37 -0600
Date: Tue, 23 May 2000 10:42:37 -0600 (MDT)
From: Jason Gunthorpe <jgg@ualberta.ca>
X-Sender: jgg@wakko.deltatee.com
To: amb@gedanken.demon.co.uk, 64530-done@bugs.debian.org
cc: APT Development Team <deity@lists.debian.org>
Subject: Re: Bug#64530: apt: The http method does not work correctly for HTTP/1.0 proxies
In-Reply-To: <[🔎] E12txUf-0000JV-00@g2.gedanken>
Message-ID: <[🔎] Pine.LNX.3.96.1000523103048.3868C-100000@wakko.deltatee.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Delivered-To: 64530-done@bugs.debian.org
On Mon, 22 May 2000 amb@gedanken.demon.co.uk wrote:
> This is the code that checks for the returned header from the server. There is
> no checking here if the server is HTTP/1.0 or HTTP/1.1. I think that you need
> the following two lines before the return statement.
>
> if(Major==0 || (Major==1 && Minor==0))
> Encoding = Closes;
None of this is really necessary, the server will close the connection on
its own if it really does not support pipelining. There is an HTTP/1.0
extension that does pipeline when there is a Connection: header present so
this test is wrong.
You are confused about what the Encoding variable does. It does not
control how APT handles pipelining, it (as it says) controls how APT
decodes the data (It is the transfer encoding). There are only 3 options,
Chunked - Variable length data with sub-headers
Stream - Valid content-length
Closes - No valid content length, closed connection == finished transfer
The only difference between Encoding=Stream and Encoding=Closes is that
closes does not check the length of the response [ie, a short response
will not be detected].
All of your patches are wrong. The correct encoding for most HTTP/1.0
responses with Content-Length headers is 'Stream'.
> Having tried these modifications on an HTTP/1.0 proxy I have noticed a
> much better performance. Previously about half of the requests to the
> server were never made (checking the server logs). I think this is
Whatever you are using for server/proxy is really busted then, it is not
following HTTP/1.0, it is not a bug in APT's method. All your changes do
is force APT to abort the connection once the first request is done
This is *particularly* true if the result is corrupt files. Any HTTP/1.0
conformant server/proxy will not corrupt data when presented with
pipelined requests.
Jason
Reply to: