[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: shutting down httpredir.debian.org?

On Thu, Apr 14, 2016 at 11:49:12AM +0200, Bastian Blank wrote:
> On Thu, Apr 14, 2016 at 09:02:18AM +0000, Peter Palfrader wrote:
> > You could argue (and I have), that that file-based redirects are not
> > ideal if your update is downloading lots of little files.  The latency
> > hit of many redirects is non-trivial.
> My last test showed problems with httpredir returning different mirrors
> with each request, which made keep-alive completely impossible.
> Downloading all the stuff during a cdebootstrap run from a real mirror
> took 5s[1], from httpredir at least 20s[2].

Well, its kinda the point of httpredir to give you different mirrors[1]
(which ideally all have the same state – if not you get the hashsum
mismatches). That allows "long living" clients like apt to talk to the
different mirrors at the same time (proper keep-alive and pipelining to
them all). One-shot clients like wget which debootstrap is using are
obviously not doing either (with the same argument its also unwise to
use a https mirror for those clients).

It is frequently said that apt should "just" open multiple connections
to the same mirror and stuff like apt-fast sells bigtime on this, but as
a categorical imperative[2] it makes no sense: Of course you are
speeding up your throughput if you add more cars to a traffic jam – but
only as long as other people don't use the same strategy. If they all do
it the traffic jam is even bigger and its actually slower for everyone.

[1] Which is not to say that it must be httpredir which gets us such
a list, but it was possible and someone implemented it.
[2] I know, browsers do it all the time – but most websites are build
dynamic requiring non-trivial time on the server to process so it kinda
makes sense as a workaround; repositories are pretty static) and the
HTTP1.1 spec actually allows it. It gets slightly more sensible with
HTTP2 now – at least server/proxy implementors can't say pipelining is
too much state keeping to implement properly anymore. ;)

Best regards

David Kalnischkies

Attachment: signature.asc
Description: PGP signature

Reply to: