Re: Too few up-to-date CD image mirrors
On Fri, 29 Nov 2002, Joey Hess wrote:
> Richard Atterer wrote:
> > Basically, the current situation WRT CD image mirrors is getting, er,
> > "irritating": Far too few of them carry the 3.0r0 images, many only
> > have the jigdo files or 2.2r6.
>
> BTW the situation seems to be most pathetic for the US mirrors for some
> reason, I have much better luck finding isos on the other mirrors.
> Cannot find any debian iso's of 3.0 in the US at all.
Yes, I have gotten the question on "can I find it anywhere closer than
Sweden" from people in the US, and not being able to answer. I can name 2
or 3 mirrors in Sweden and a bunch more in other european countries
though.
> > Originally, the plan for 3.0r0 was that ISOs would be made available
> > via jigdo and that mirrors would update using the "jigdo-mirror"
> > script which I wrote especially for that purpose. (And I mailed all
> > mirror admins with the details.) Later, the images would also become
> > available "normally" via rsync (AFAIR), but somehow that never
> > happened.
> >
> > But few admins seem to have set up jigdo-mirror... I think for most of
> > them, mirroring something is not an option if it can't be fetched with
> > http, ftp or rsync. <shrug>
Well, even for secondary mirrors one of our mirrors found a problem with
some version of a debcdmirror script that requires an ls-lR file that we
do not have.
> > So let's do it their way: I propose that we offer rsync (and maybe
> > HTTP?) access to the images again starting with 3.0r1. raff has enough
> > disc space, so the size should not be a problem.
Well, when it comes to larger mirrors (as in number of diffrent archives
they carry), compiling/installing new software for a specific archive is
probably considered a bit too much work. I see jigdo as a temporary way to
get the jigdo files from the main server before the isos are made
available a week later or so. To avoid the huge congestion at release
time. I would guess at about 10-20 mirrors in total, but that should be
enough to get the isos out in resonable time after a release.
Also this kind of change takes time. If you get no user inquiries about
"why hasn't $foo updated for the last couple of months", chances are that
you'll never notice unless the mirroring script starts logging errors.
> I offered the web team to write a program to check the cd image mirror
> list, and output some kind of table of what ftp mirrors:
>
> - are down or broken
> - have jigdo
> - have current isos
>
> With the idea being this would be used to prune the list shown to users
> on the web site or perhaps generate a seperate list for people who get
> to the mirror list wanting jugdo files, vs. those who get to the list
> looking for an iso mirror. I think it's important to automate this.
Yes, this would be very good. I have been thinking of writing such a
script myself, but never found the time for it. I would be very grateful
if you could write such a script.
> I will not try to check the http mirrors though. Indeed, I think the
> http mirrors are a bad idea, since (as is seen in my mistake reading one
> of them in the gentoo thread), they introduce a bunch of third party web
> sites of varying quality that users must navigate to find isos. I hope
> the http stuff is not a necessary evil.
For us http is just a better ftp, without the "logging in as anonymous"
part. The standard apache file listing is good enough.
> Once the mirror list is whipped into shape, it'll also be possible to do
> things like a cgi that uses it to pick a mirror based on a user's
> address, for one-click iso downloading.
Could be nice, but the most important thing is to be able to separate the
list into chunks with the top one being the mirrors carrying the latest
release.
> Or course it would be best if I had a canoical site to run the checking
> program against, then it could just do a straightforward comparison.
> Unfortunatly, it seems that cdimage.debian.org no longer carries isos,
> and so there is no canoical site? Is that right?
That has been a main issue for getting other mirrors to mirror the images
too. "They aren't on cdimage.debian.org, so why should we bother with
some kind of not as official ones." The fact that we carry isos with
released md5sums seems not as important. Or just a convenient excuse to be
a bit more lazy.
/Mattias Wadenstein
Reply to: