Re: apt: HTTP transfer method does not use available bandwidth
[I originally thought this was a problem with apt, but it appears to be a
problem with HTTP transfers in general, so I am putting it on the devel list.]
Jason Gunthorpe wrote:
>
>> The HTTP transfer method now used by apt is slow and does not use all the
>> bandwidth available. By comparison, the ftp method uses all available
>> bandwidth.
>
>First off, you are using unix.hensa.ac.uk for both of your tests?
Yes. (Except for http://ftp.de.debian.org/debian-non-US for the non-us
section.)
>
>I just performed a quick comparision here, I downloded the 1M file,
> ftp://unix.hensa.ac.uk/mirrors/debian/hamm/hamm/binary-i386/Packages
> http://unix.hensa.ac.uk/mirrors/debian/hamm/hamm/binary-i386/Packages
>
>I used lftp for the ftp method and apt's http method for http. lftp took
>44 seconds (25.2 k/s) to download the file and the http method took 31
>seconds (35 k/s) - this includes the time to establish the connection and
>send the request while ftp only included the time from first byte recived.
>
>As you can see not only is the bandwidth higher for http but the total
>overall transfer-rate is also much higher than ftp.
OK, so I shouldn't be having a problem...
>
>Just for kicks I tried the same with wget, it got 27.60 kb/s on http and
>30k/s on ftp (As you can see my connection to the UK varies quite a bit)
I'm IN the UK; the very best I can get is 8k/s (one ISDN channel).
>I would try wget on the above two URL's and see what you get for speed,
Recovering the same file, while apt is transferring data:
unix.hensa.ac.uk sunsite.doc.ic.ac.uk
wget http: 1.97KB/s 1.90KB/s
wget ftp: 5.19KB/s 5.42KB/s
ftp: 4.2 Kbytes/sec 4 Kbytes/sec
(`wget URL' as opposed to `get file' inside an ftp session.)
OK, so it's not apt, but http.
> if
>http is slower then perhaps your ISP is transparently running you through
>a proxy server - also check that you do not have http_proxy in your
>environment, you might be using an overloaded proxy server. (In which case
>complain to your ISP :> )
Should running squid locally make a difference? (I shut it down, but it
doesn't
seem to have changed anything.) My ISP has a proxy-server, but you have to
configure Netscape to use it, so presumably it is not transparent.
I'm trying again with dselect+apt just now. This time I let it go past getting
the packages files; it certainly seems to show a denser pattern on the monitor
when getting the actual .deb files, though still not the solid black that I
expect
to see from ftp...
...now it's gone back to the original light pattern.
Is there anything about the HTTP protocol which makes a difference if you are
using
a lower bandwidth? For example, if I cannot accept stuff at the rate at which
the other end can push it out, will the other end reduce its attempted output
rate?
What packages are involved that might need investigating?
--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
PGP key from public servers; key ID 32B8FAA1
========================================
Come to me, all you who labour and are heavily laden, and I will
give you rest. Take my yoke upon you, and learn from me; for I am
meek and lowly in heart, and you shall find rest for your souls.
For my yoke is easy and my burden is light. (Matthew 11: 28-30)
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Reply to: