Re: APT do not work with Squid as a proxy because of pipelining default

[Goswin von Brederlow <goswin-v-b@web.de>]

Bjørn Mork <bjorn@mork.no> writes:

> Petter Reinholdtsen <pere@hungry.com> writes:
>> [Roger Lynn]
>>> But apt has been using pipelining for years. Why has this only just
>>> become a problem?
>>
>> It has been a problem in Debian Edu for years.  Just recently I
>> figured out the cause and a workaround.
>
> And FWIW I have experienced this problem for years too, but never
> figured out why until this discussion came up.  And I do want to claim
> more than common user knowledge of http proxy servers.  Still, it never
> occured to me that my spurious apt problems could be caused by proxies.
> And no, it's not just squid - I've been seeing the exact same at work
> where the office network have some intercepting proxy solution from
> websense.
>
> Anyway, this is definitely the type of problem that can and do exist for
> years without that necessarily causing a massive number of bug reports
> against apt.  I still do not think that is an argument against fixing
> it?
>
> Can we please agree that in the real world
> 1) RFC1123 beats any other standard: "Be liberal in what you accept, and
>    conservative in what you send", and
> 2) http proxy servers cannot always process pipelined requests due to
>    the complexity this adds (complexity is always bad for security), and
> 3) http clients cannot know whether their requests are proxied
> ?
>
> The sum of these three points is that a http client should never send
> pipelined requests.  

Wrong, at least from arguments 2 and 3.

A HTTP/1.1 conforming server or proxy is free to process pipelined
requests serially one by one. The only requirement is that it does not
corrupt the second request by reading all available data into a buffer,
parsing the first request and then throwing away the buffer and thereby
discarding the subsequent requests in that buffer. It is perfectly fine
for the server to parse the first request, think, respond to the first
request and then continue to parse the second one.

Note that that behaviour in the server already gives a huge speed
increase. It cuts away the round trip time between the last responce and
the next request. For static web content the speedup of processing
pipelined requests truely in parallel is neglible anyway. Only dynamic
pages, where formulating the responce to a request takes time, would
benefit by working on multiple responces on multiple cores. And those
are probably busy handling requests from other connections. So I
wouldn't expect servers to actually to parallel processing of pipelined
requests at all.


And your argument 1 applies perfectly to fixing squid by the way. It
should accept pipelined requests and then it can proces them one by one
and send them non pipelined if it likes. It should NOT corrupt the
requests/responces.

So just from your argument 1 APT should default not to pipeline and
squid should be fixed.

MfG
        Goswin