Re: mirror.debian.net maintenance
On Sat, Mar 29, 2008 at 01:41:27AM +0100, Leo costela Antunes wrote:
> > * I don't see how pick_best_mirror() ever does any load balancing, when
> > it will always return the same mirror (the one with the best score),
> > given the same input variables?
>
> Actually I hadn't thought about this. My original idea was just
> returning the right ftp.<cc>.d.o for the originating country, but now
> that you mention it, we could merge our works in a much better way: We
> can scrap a huge chunk from my script and just leave the functionality
> to resolve and point to the mirror.d.n structure.
> As you could see, my script isn't doing any incredible heuristics, just
> some very simple scoring which I'm not even sure are of any help to
> mirror admins (the whole prioritizing leafs over push-primaries thing),
> so pointing to a big round-robin list of mirrors in that country should
> be enough, and you've got that part easily set up.
> Plus, we can benefit from your virtual host filtering! :)
Hm, yeah. I'm thinking what would be the best way to handle this selection
in general. The country can be autodetected remotely, but the architecture
can be autodetected locally. We could put your stuff at
<arch>.{official,unofficial}.mirror.debian.org (.net during testing),
and extend the sources.list(5) parser to support that somehow - we
could possibly use the $(ARCH) variable reference, but that seems to
be unavailable in the host parameter right now, and it's also ugly.
Instead, something like
deb arch+http://official.mirror.debian.org/debian stable main
And then it prepends the architecture string to the hostname.
> >> That's precisely my intention, but since I'm currently using PDNS, I'm
> >> afraid the DSA folks will have some sort of problem with it. But I guess
> >> it can't hurt to ask! :)
> >
> > I'm fine with pdns, so I can help you test and maybe convince them :)
>
> Thanks, how can we kick this off then?
Can you verify if your script can run on stable?
> I've included a patch for mirror_list.pl to generate a complete list of
> mirrors for an entire arch, no matter the country (I inverted the ${cc}
> and ${arch} order in the %zone_entries hash to be able to do it in the
> same loop structure).
> The idea is to use it as a last case default if the geomirror script
> can't find the country, instead of hard-coding a default.
> I'm not sure if it's a great idea since it duplicates the number of
> entries, but if you think it's ok, I'd say it's better then the alternative.
I'd rather if this was done in a single separate loop at the end, it would
seem a bit cleaner that way...?
I'm also a bit worried about the nsupdate buffer limit, I'll need to test
it.
> Since the TTL on the mirrors is about 4 hours, couldn't the interval
> between cron zone updates be lowered to perhaps twice a day, at least? I
> figure it would lower the chances of people stumbling into dead mirrors
> once they're temporarily - or otherwise - removed from the masterlist.
Oh yeah, sure, actually they get changed even if they aren't removed from
the master list - their IP addresses can change in the meantime.
The script takes about 3:15 to run, so it's not a problem to make it run
several times a day.
Another problem that I should mention here - read timeouts also get treated
as a disqualification. That probably shouldn't be done that way - we could
retry. Or use the data from DMC, or data from Mole. It won't help announced
downtimes, but round-robins should take care of that. That could also be
checked... oh the possibilities :)
--
2. That which causes joy or happiness.
Reply to: