Re: APT do not work with Squid as a proxy because of pipelining default
David Kalnischkies <kalnischkies+debian@gmail.com> writes:
> Hi all,
>
> i don't want to interrupt your battles so feel free to ignore me,
> but i want to raise some questions (for you and me) none the less:
>
> The notice about the - in the eyes of the writer of this manpage
> section - broken squid version 2.0.2 in the apt.conf manpage
> was changed the last time in 2004, so the issue isn't "new".
> The manpage at least claims that this squid version is broken
> also in respect to other cache control settings.
>
> I don't know a single bit about squid but a search for "squid pipeline"
> turns up some documentation about a pipeline_prefetch setting:
>> Available in: 3.1 2.7 3.HEAD 2.HEAD 3.0 2.6
>>
>> To boost the performance of pipelined requests to closer
>> match that of a non-proxied environment Squid can try to fetch
>> up to two requests in parallel from a pipeline.
> http://www.squid-cache.org/Doc/config/pipeline_prefetch/
>
> For somebody without knowledge this looks like as any
> version in debian should be able to handle a pipeline -
> otherwise this setting wouldn't make much sense?
>
> The default value for the APT option above is btw 10 and in apt
> #413324 we have a claim that squid works well with a value of 5
> or even 8 -- so it is maybe "just" a bug in handling "too much"
> pipelined requests? Or something comparable to what happened
> in #541428 regarding lighttpd and pipelining (timeout)?
> (i am just shooting into the dark)
>
>
> Also, then we talk here about pipelines and her usage
> keep in mind that APTs http usage is special compared to
> an implementation and usage in a browser:
> We have a trust chain available so we should be on the save
> side security wise, the number of debian archives is limited
> and most of them should be on a sane webserver
> (if not i would not have much trust in the archive?) and
> especially on "apt-get update" we have either a lot of cache
> hits (file has not changed) or a lot of very small files (Release,
> Index and maybe pdiff) to transfer. New package updates come
> from the same archive most of time and most packages are
> relatively small, too, but having an upgrade including at least
> 500 packages is relatively common?
Do I hear an idea in there? Use pipelineing for small stuff (header
check, Release, Release.gpg, pdiff, dsc) but then default to depth 1 for
the big files (Packages.gz, debs, orig.tar.gz).
> On the other hand APTs http client isn't as nice as he could be in
> terms that he could fallback to non-pipeline, retry or whatever.
> (and i wouldn't be too surprised if this would turn out to be an APT bug)
> As we all know APT is a debian native tool and the base of a whole
> bunch of other stuff so beside ranting about his shortcomings we
> could also work on patches as the people with enough knowledge
> to do this seems to be already around in this thread.
>
>
> Thanks in advance and best regards,
>
> David Kalnischkies
>
>
> P.S. Sry Luigi Gangitano for cc'ing, but i don't know if you follow
> the thread and i included too often "squid" in the mail to not direct
> the mail into your direction.
MfG
Goswin
Reply to: