[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: state of the cdrom mirrors report



I've more or less funished up the program to check the status of our ftp
cdrom mirrors. My program generates a Mirrors.masterlist.status file,
with some fields to indicate the state of the mirrors. The fields are
chosen so we can run this program every few days, and stop pointing to a
mirror if it has been down for a given time period.

And I've modified mirror_list.pl to use the new fields to generate a list
that has only known-good mirrors on it. I took the approach of assuming
that a site's http mirror is good if its ftp mirror checks out ok.

As an example, I've attached a version of the cd image mirror list web
page generated with the above.

If all this looks good, I'll need help checking it into cvs, as I have
not had access to the website cvs since I stopped authoring DWN. I have
attached a diff against cvs for my changes, and a current copy of
check_cd_mirrors.pl and of Mirrors.masterlist.status (which should both
be put into webwml/english/mirror/).

I think we should set up a cron to run the check_cd_mirrors.pl every day
(or every 3 days, but it's not very expensive to run, though it does
take about an hour to complete) on gluck, and commit its changes to the
Mirrors.masterlist.status (which can be done on gluck best since the cvs
repository is there).

-- 
see shy jo
? english/mirror/Mirrors.masterlist.status
? english/mirror/check_cd_mirrors.pl
Index: english/CD/http-ftp/index.wml
===================================================================
RCS file: /cvs/webwml/webwml/english/CD/http-ftp/index.wml,v
retrieving revision 1.15
diff -u -r1.15 index.wml
--- english/CD/http-ftp/index.wml	16 Oct 2002 20:30:45 -0000	1.15
+++ english/CD/http-ftp/index.wml	30 Nov 2002 23:55:29 -0000
@@ -41,10 +41,12 @@
 <hr>
 
 <h2><a name="stable">Official CD images of the "stable" releases</a></h2>
-
+<!-- 
 <p>Note that <strong>some mirrors are not up to date</strong> -
 before downloading, check the version number of the images is the
 same as the one listed <a href="../#latest">on this site</a>!</p>
+-->
+<!-- They are now! - JEH -->
 
 #use wml::debian::countries
 #include "$(ENGLISHDIR)/CD/http-ftp/cdimage_mirrors.list"
Index: english/distrib/netinst.wml
===================================================================
RCS file: /cvs/webwml/webwml/english/distrib/netinst.wml,v
retrieving revision 1.6
diff -u -r1.6 netinst.wml
--- english/distrib/netinst.wml	25 Aug 2002 20:37:58 -0000	1.6
+++ english/distrib/netinst.wml	30 Nov 2002 23:55:30 -0000
@@ -14,9 +14,9 @@
 <p>There are two options for installs over the network:</p>
 
 <ul>
-  <li><a href="../CD/netinst/">Minimal CD</a>:
-  Instead of getting a full 650MB CD image, you just download a CD image
-  file which contains the bare essentials necessary to install the rest.
+  <li><a href="../CD/">CD</a>:
+  You download a CD image file. Minimal CD images are available, as is
+  a fast and efficient means to download a full CD from a mirror near you.
   For the moment, it's necessary to have access to a CD recorder in order
   to use this.</li>
   <li><a href="floppyinst">Floppy disks</a>:
Index: english/mirror/mirror_list.pl
===================================================================
RCS file: /cvs/webwml/webwml/english/mirror/mirror_list.pl,v
retrieving revision 1.44
diff -u -r1.44 mirror_list.pl
--- english/mirror/mirror_list.pl	24 Nov 2002 18:50:19 -0000	1.44
+++ english/mirror/mirror_list.pl	30 Nov 2002 23:55:37 -0000
@@ -477,8 +477,9 @@
     foreach $site (sort @{ $countries{$country} }) {
       (my $countrycode = $country) =~ s/^(..) .*/$1/;
       if ($which eq "httpftp") {
-        if (defined $mirror{$site}{method}{'cdimage-ftp'} ||
-            defined $mirror{$site}{method}{'cdimage-http'}) {
+        if (defined $mirror{$site}{method}{'cdimage-ftp'} &&
+	    (! defined $mirror{$site}{status}{'cdimage-ftp-status'} ||
+	     $mirror{$site}{status}{'cdimage-ftp-status'} eq 'good')) {
           print "<li><${countrycode}c>: $site: ";
           if (defined $mirror{$site}{method}{'cdimage-ftp'}) {
             print <<END;
@@ -913,7 +914,7 @@
 	exit;
 }
 
-open SRC, "<$mirror_source" ||
+open(SRC, "<$mirror_source") ||
   die "Error: problem opening mirror source file, $mirror_source\n"
      ."Use the -m option?\n";
 
@@ -944,6 +945,23 @@
 }
 if ($current ne "") {
 	process_line($current);
+}
+
+open(STATUS, "<$mirror_source.status") ||
+  die "Error: problem opening mirror status file, $mirror_source.status\n"
+     ."Use the -m option?\n";
+{
+	local $/="\n\n";
+	while (<STATUS>) {
+                if (/Site:\s+(.*)/i) {
+                        my $site=$1;
+                        foreach my $line (split("\n", $_)) {
+                                my ($key, $value) = $line =~ /^(.*): (.*)/;
+				$key=lc($key);
+				$mirror{$site}{status}{$key}=$value;
+			}
+		}
+	}
 }
 
 # count the number of mirrors
Site: antirix.softnet.tuc.gr
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038694000

Site: carroll.aset.psu.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691559

Site: debian-cd.rutgers.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690879

Site: debian.ankara.edu.tr
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690826
CDImage-ftp-lastgood: 1038690826

Site: debian.attica.net.nz
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691778

Site: debian.balt.net
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038693633

Site: debian.blueyonder.co.uk
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038692014

Site: debian.efis.ucr.ac.cr
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038696001

Site: debian.ens-cachan.fr
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038689472
CDImage-ftp-lastgood: 1038689472

Site: debian.essentkabel.com
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038691895
CDImage-ftp-lastgood: 1038691895

Site: debian.fastweb.it
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690753

Site: debian.fifi.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038692617

Site: debian.lami.univ-evry.fr
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038692149

Site: debian.linux.org.tw
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038692026

Site: debian.oregonstate.edu
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038693957
CDImage-ftp-lastgood: 1038693957

Site: debian.sh.cvut.cz
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691546

Site: debian.tod.net
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695875

Site: debian.uchicago.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690838

Site: debian.udsu.ru
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690105

Site: debian.uni-essen.de
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038692011
CDImage-ftp-lastgood: 1038692011

Site: ftp-linux.cc.gatech.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690114

Site: ftp-mirror.internap.com
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690356

Site: ftp-stud.fht-esslingen.de
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038693355
CDImage-ftp-lastgood: 1038693355

Site: ftp.au.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690971

Site: ftp.belnet.be
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038692158
CDImage-ftp-lastgood: 1038692158

Site: ftp.bittivuoto.net
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695881

Site: ftp.cl.debian.org
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690762
CDImage-ftp-lastgood: 1038690762

Site: ftp.cs.stevens-tech.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690747

Site: ftp.cvt.stuba.sk
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690922
CDImage-ftp-lastgood: 1038690922

Site: ftp.de.debian.org
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038693620
CDImage-ftp-lastgood: 1038693620

Site: ftp.du.se
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690889
CDImage-ftp-lastgood: 1038690889

Site: ftp.ee.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690915

Site: ftp.egr.msu.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690948

Site: ftp.es.debian.org
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038691944
CDImage-ftp-lastgood: 1038691944

Site: ftp.esat.net
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690896
CDImage-ftp-lastgood: 1038690896

Site: ftp.eu.uu.net
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690353
CDImage-ftp-lastgood: 1038690353

Site: ftp.fi.debian.org
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690343
CDImage-ftp-lastgood: 1038690343

Site: ftp.fr.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691701

Site: ftp.funet.fi
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690080
CDImage-ftp-lastgood: 1038690080

Site: ftp.hk.debian.org
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690980
CDImage-ftp-lastgood: 1038690980

Site: ftp.hr.debian.org
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690939
CDImage-ftp-lastgood: 1038690939

Site: ftp.hu.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689752

Site: ftp.icm.edu.pl
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691862

Site: ftp.ie.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038693923

Site: ftp.iglu.org.il
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691954

Site: ftp.iinet.net.au
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691371

Site: ftp.irb.hr
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690836

Site: ftp.is.co.za
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690944

Site: ftp.jyu.fi
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690801

Site: ftp.keystealth.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691223

Site: ftp.kreonet.re.kr
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038692152

Site: ftp.leo.org
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695702

Site: ftp.linux.tucows.com
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695708

Site: ftp.lug.udel.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690982

Site: ftp.matrix.net.br
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690952

Site: ftp.minet.net
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038693637

Site: ftp.mpi-sb.mpg.de
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690929
CDImage-ftp-lastgood: 1038690929

Site: ftp.nectec.or.th
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691418

Site: ftp.nerim.net
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038692528

Site: ftp.nl.debian.org
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690833
CDImage-ftp-lastgood: 1038690833

Site: ftp.no.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689756

Site: ftp.nrc.ca
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038694812

Site: ftp.ntua.gr
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691814

Site: ftp.nuri.net
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691413

Site: ftp.oleane.net
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690750

Site: ftp.pl.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690901

Site: ftp.proxad.net
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691788

Site: ftp.psn.ru
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690336
CDImage-ftp-lastgood: 1038690336

Site: ftp.rediris.es
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038693630
CDImage-ftp-lastgood: 1038693630

Site: ftp.rutgers.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038694137

Site: ftp.se.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689481

Site: ftp.si.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689598

Site: ftp.students.cs.unibo.it
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038689467
CDImage-ftp-lastgood: 1038689467

Site: ftp.sun.ac.za
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690877
CDImage-ftp-lastgood: 1038690877

Site: ftp.sunet.se
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038693982

Site: ftp.telepac.pt
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689590

Site: ftp.ticklers.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690846

Site: ftp.tiscali.be
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690906

Site: ftp.tiscali.de
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038692446
CDImage-ftp-lastgood: 1038692446

Site: ftp.tiscali.nl
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690304

Site: ftp.tku.edu.tw
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690796
CDImage-ftp-lastgood: 1038690796

Site: ftp.tu-chemnitz.de
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690379
CDImage-ftp-lastgood: 1038690379

Site: ftp.tu-clausthal.de
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038691405
CDImage-ftp-lastgood: 1038691405

Site: ftp.u-strasbg.fr
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038691565
CDImage-ftp-lastgood: 1038691565

Site: ftp.unina.it
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691373

Site: ftp.wa.au.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038693649

Site: ftp.zcu.cz
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038689747
CDImage-ftp-lastgood: 1038689747

Site: ftp2.de.debian.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689765

Site: ftp2.it.debian.org
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695462

Site: gajah.vlsm.org
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695582

Site: gd.tuwien.ac.at
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038691760
CDImage-ftp-lastgood: 1038691760

Site: iso.linux.hr
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689504

Site: kebo.vlsm.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691557

Site: koyanet.lv
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695461

Site: linorg.usp.br
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691837

Site: linux.cdpa.nsysu.edu.tw
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038691398
CDImage-ftp-lastgood: 1038691398

Site: linux.csie.nctu.edu.tw
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695848

Site: linux.csua.berkeley.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690946

Site: linux.inf.elte.hu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689594

Site: linux.org.by
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691221

Site: mi.mirror.garr.it
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038695808

Site: mirror.aarnet.edu.au
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038696001

Site: mirror.cict.fr
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690799

Site: mirror.cs.wisc.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691226

Site: mirror.cse.unsw.edu.au
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038693659

Site: mirror.csit.fsu.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690317

Site: mirror.datafast.net.au
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689587

Site: mirror.pacific.net.au
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689500

Site: mirror.pudas.net
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690347

Site: mirrors.kernel.org
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690882

Site: mirrors.sunsite.dk
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689897

Site: mirrors.xmission.com
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690963

Site: sajino.terra.com.pe
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038689602

Site: sunsite.informatik.rwth-aachen.de
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690913

Site: sunsite.ualberta.ca
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690844

Site: toxo.com.uvigo.es
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690745

Site: www.mirror.ac.uk
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690961
CDImage-ftp-lastgood: 1038690961

Site: zagloba.t19.ds.pwr.wroc.pl
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690287

Attachment: check_cd_mirrors.pl
Description: Perl program

Attachment: pgpKc5UNHRlL5.pgp
Description: PGP signature


Reply to: