Re: Debian ISOs
On Thu August 31 2006 00:27, Subredu Manuel wrote:
>> Bruce Sass wrote:
>> When a new release is made, all servers (and mirrors) are getting
>> hit. First, the tier 1 (ftp.<country>.debian.org), and then the other
>> mirrors.
>
>Would each mirror or region need a unique metalink file to ensure that
>happens?
>
>I'm thinking that if all clients see the same metalink file they will
>all hit the same servers, and someone wanting to support the local
>mirrors could start fetching from 1/2-way around the world (I suppose
>there would still need to be non-metalink access just for this reason.)
>
>Hmmm, that brings up another question. Each downloadable file would need
>its own metalink file. How much space will that take and at what point
>would it become significant.
>
>> So, if you only consider http and ftp, metalink is the
>> logical choice, since it switches automatically to a new server, when
>> the rate drops (at least, it should, but this depends on the client).
>
>Transfers in progress could change servers, or new clients would get a
>different server?
>
>The former is what I would be concerned about (additional overhead,
>exacerbated by poorly coded or configured clients); the latter would be
>a good thing but could also be done with a smart round-robin setup
>(maybe already is via http.us.debian.org.)
Hi Bruce, just wanted to say thanks for investigating Metalink. These are
all valid concerns. For the last few months, the only big user of Metalinks
has been OpenOffice.org, and I haven't heard any complaints from
mirror/server admins. That doesn't mean there aren't any. I'm sure there
will be problems, and I'd like to work them out as soon as possible.
Metalink allows for information on the country location ("location") of a
server along with priority ("preference"):
<url type="ftp"
location="au"
preference="30">
ftp://mirror.pacific.net.au/OpenOffice/stable/2.0.3/OOo_2.0.3_LinuxIntel_ins
tall.tar.gz
</url>
The simplest way might be for clients to try local mirrors, in order of
preference, then all other mirrors in order of preference (or, other close
countries). I think this added information of location and priority will
lead to more efficient use of bandwidth over just plain links (hopefully).
Manuel has also made a web interface that generates Metalinks depending on
what country you select.
Metalinks can list multiple or single files.
The size of Metalink files depends on how much information you want to
include and how many mirrors are listed. For example, Metalinks listing
every kernel.org or OOo mirror are around 20k. All mirrors for Fedora ISOs:
54k, Ubuntu ISOs: 58k, SUSE ISOs: 10k.
I'd be interested in working on any things that may be an issue for Debian
as I'm sure they would affect others too.
One thing I'd definitely like to do is get aria2
(http://aria2.sourceforge.net/) included in Debian. It's a BitTorrent and
Metalink command line client. Maybe recommending certain clients over
another and working with the authors of those clients to be correctly coded
and configured by default could help.
Anthony Bryan
Metalink [ www.metalinker.org ]
Reply to: