Re: ftp1.us.debian.org and FTP mirrors
On Thu, Oct 22, 1998 at 07:03:11PM +0200, Lukas Eppler wrote:
> Once someone posted this script to find the fastest debian mirror, which
> could be used in some way in the postinst of apt:
>
> #!/bin/sh
> # Script to figure out the fastest Debian mirror sites
> zgrep "\..*:/" /usr/doc/mirror/README.Debian-mirrors.gz | \
> awk '{ print $1 };' | \
> awk -F: '{ print $1 }' | fping -ae | \
> awk '{ print substr($2,2),$1 }' | sort -n| head -10
Hmm, interesting. I wasn't aware of the fping command. Something about it
is slow, however. I suspect sequential DNS lookups to be the culprit.
tcpdump shows virtually no network traffic for most of the time fping is
running, and then large bursts later on.
fping-script results: (time: 4m34.887s)
32.7 sunsite.unc.edu
37.1 llug.sep.bnl.gov
42.7 ftp.debian.org
101 linux.eecs.umich.edu
105 ftp.cdrom.com
129 ftp.leidenuniv.nl
134 ftp.belgium.eu.net
138 ftp.fuller.edu
144 ftp-nog.rutgers.edu
149 debian.ssc.com
Using the following script with netselect:
#!/bin/sh
zcat /usr/doc/mirror/README.Debian-mirrors.gz \
| grep "\..*:/" | awk '{print $1}' | sed 's/:.*//' \
| xargs netselect -v -s 10
I get these results: (time: 0m37.547s)
84 llug.sep.bnl.gov
93 debian.crosslink.net
115 ftp.debian.org
204 groucho.cs.wisc.edu
299 linux.eecs.umich.edu
312 sunsite.unc.edu
330 ftp.cdrom.com
334 ftp.caldera.com
350 ftp.fuller.edu
396 ftp.oro.net
Ignore the actual scores produced by the programs; they use different rating
techniques. netselect takes into account packet loss and hop count in its
calculations.
Both benchmarks are after seeding the local BIND nameserver with all DNS
names in the list. Both programs take longer if you don't do that, but
netselect does parallel DNS lookups so it's the faster of the two. For a
valid comparison, be sure to check whether your BIND nameserver has cached
all the names internally. Remember that most people will only run the
script once, so realistically DNS names won't be cached.
Both programs are much faster if we remove broken DNS names from the mirrors
list.
In the case that there are no dead DNS servers in the list and your local
DNS server has all valid names cached, netselect is slower than fping. This
seems to be because netselect pings each host 10 times by default, which
fping just does them once. You can add '-t 2' or '-t 1' to netselect to
reduce the number of pings at the expense of some accuracy in the results.
(eg. the packet loss calculation is pretty useless that way). Netselect is
then 2x faster or so.
Empirical evidence has shown that for me, the top three or four servers
chosen by both programs are all approximately the same speed so it's
understandably difficult to pick the "very best" one using only a few ping
packets.
Have fun,
Avery
Reply to: