Re: some proposed changes/requirements for apt
On Thu, 16 May 2002, Jason Gunthorpe wrote:
> > traffic levels (about 1Tb/day) and we have to take measures to deal
> > with this. Splitting it between different types of servers has worked
> > well, but the only problem we have is dealing with apt.
>
> 1 terabit? Hmm.. ftp.d.o does about 3 terabit/day and it isn't bothered
> too much running apache. It's just a little P2-350 too.
>
> If you are doing a terabyte/day.. Well.. good lord! :>
Tbyte, yes. /whack self for not remember TB vs Tb. Obviously not all of
that is debian, but it's a significant percentage now.
> Anyhow, if you are worried about load from apt then doing redirects at all
> will be noticable both to you and your clients. It is much better to
> simply avoid them entirely.
in practice it hasn't worked out that way. the overhead of the redirect
has been significantly less than the overhead of apache to process the
request.
since we swapped to using apache+boa, we've seen load averages drop from
around 200-300 to 5-20 in an average day. the primary offendors seem
to be a combination of apache trying to process iso downloads, but also
getright and other accelerators doing stupid things (like downloading
a 650M iso in 10K chunks.. 65,000 requests hitting the server as quickly
as the client can send them).
> > with a browser will fail. (i don't even pretend to be clueful about
> > using apt - does it ever try and do any kind of directory listing?)
>
> Nope. Only http get. One big warning though, if you are using a web server
> other than apache it may be slightly buggy and people using it with APT
> may get upset. In the past servers have had problems with HTTP/1.1
> If-Range, pipelining and keepalive that APT makes heavy use of.
hmm, i'll have to chat to the boa author about this.
at the moment, my primary concern is still to figure out some way of
dealing with lack of 302 support for .gz file fetches by apt. i don't
suppose you feel like just deciding to not have any .gz files in
the debian archive instead (i.e just leave Packages* everywhere
uncompressed) :-)
regards,
-jason
--
To UNSUBSCRIBE, email to deity-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Reply to: