Hey all, First, the executive summary for mirror operators reading this: we'll be switching the primary mirror stuff for Debian to be for a small number of architectures rather than all of them; initially this will just be i386, but will probably expand to include amd64. Single architecture mirrors appear to need about 60GB of disk space, dual architecture mirrors will need about 80GB of disk space. A full mirror at the moment requires slightly over 170GB. We'll be trying to get in touch with you all further over the next week or two to make sure the changes go as smoothly as possible. Second, there's a call for help down below. Particularly for people with some web programming skillz. Okay, so! You've probably heard about this mirror split thing, also known as "scc", "second class citizens", and "the evil plot by the cabal to make poor blighted amd64 developers and users suffer unduly". At present, most mirrors mirror all the archive, and at present almost all Debian users that use those mirrors use i386 . But with the archive growing almost daily (recently breaking the 170GB mark, up from 130GB in July iirc), and mirrors often finding it hard to keep up, that's a bad tradeoff. That tradeoff becomes even worse when we're unwilling to add interesting new architectures because of the immediate increase in archive size they cause, in addition to the ongoing accretion. So obviously that tradeoff's changing in favour of partial mirrors, particularly by architecture. People have been able to do partial mirrors for a while now, and the anonftpsync  tool we offer includes explicit support for that. In further support of partial mirrors, we'll be doing these things: (a) allowing and in some cases encouraging official mirrors to mirror a limited number of archictures (b) limiting ftp.debian.org to i386 only (probably to also include amd64, depending on how popular that architecture actually is) (c) providing simple recommendations on running arch-specific mirrors (d) providing <cc>.<arch>.mirror.debian.org aliases to make it easy for users to find a local mirror that supports their preferred architecture As well as allowing additional architectures, these changes should make it plausible to think about a few other things, notably including additional suites in the archive (such as "volatile" or "backports"), and having the archive pulse occur more frequently than daily. But one of the things all this stuff requires is some good communication with our mirrors. That can use some work at the moment, and one thing that would be particularly helpful is some better tracking of who's mirroring what. At present we have the "Mirrors masterlist" , the mirrors pseudopackage on the BTS  (and its corresponding mailing list, which has public archives on master ) and the Debian mirror checker . What would be helpful would to improve our tracking stuff so that: (a) the mirror checker can provide more detailed information on mirror stability such as that offered by the apache tracker  (b) the mirror checker script can verify which architectures a mirror actually carries (c) there's some easy way for mirror admins to add, remove and update their details in the masterlist, rather than waiting for a developer to review the changes (with additions confirmed automatically by the checker, ideally) (d) there are status fields to indicate whether the mirror supports pushing downstream mirrors by ssh trigger, in the same manner as the top level Debian mirrors (e) what mirror each mirror mirrors from (ideally checked automatically by looking in project/trace; and ideally with a pretty graph generated from the data, highlighting mirrors that are out of date) (f) whether the mirror is able to accept *.mirror.debian.org requests, so that if, eg, "at.alpha.mirror.debian.org" goes down, it can automatically be pointed at another site in Austria, or if none are available, another nearby alpha mirror. But that's not really something I'm good at, and the existing folks working on organising the mirrors haven't had time to magic it up either, so if other folks would like to try their hand at whipping something up that can be made official that'd be great. Having it be something that can be run with minimal priveleges, not require PHP or a large framework, something that supports the existing Mirrors masterlist format for input and output, and be something that is written to run efficiently would be great. Naturally it needs to be free software. Followups on #debian-mirrors on OFTC, or to -devel I guess. This isn't a showstopper, but I think it's fair to say everyone will benefit from us doing a better job here than we currently are, so additional help would be _really_ useful. What else? As of today, there are some file lists in /debian/indices/files that you can use to help decide what stuff to download. In particular, you can download an i386 only mirror by simply invoking rsync as: rsync -av --progress --delay-updates -a --files-from :indices/files/arch-i386.files \ --delete --delete-after --max-delete=1000 \ rsync://MIRROR/debian/ ./ Likewise for other architectures. Note that the hurd-i386 files list currently includes source and arch:all packages from oldstable, stable and testing as well as unstable, which is a bug. Note that this will mean that if you're running a non-i386 architecture, you may find your preferred mirror will stop working when all this actually happens *including for stable users*. There'll be another announcement before this actually happens. On a related note, we may be moving oldstable onto archive.debian.org before ceasing security support, either for woody or shortly after future releases (possibly on the order of a month after). Note that point releases of oldstable are not undertaken, and security updates are thus not included directly into oldstable -- at present it's mostly maintained to ensure security buildds can download build-depends, which can happen equally well via archive.debian.org.  Note that the above isn't final, but it's hopefully pretty close to being so. I hope that's all for now. :) Cheers, aj  The stats for ftp.debian.org over http is about 96.4% of users using i386 exclusively, 1.4% wanting powerpc, 0.5% wanting sparc, and the remaining architectures getting about .2% each -- stats are by IP looking at queries for Packages files, and calculated over a week with around 144,000 unique IPs seen in that time. Beyond the 96.4% wanting i386 exclusively, and additional 0.8% wanted i386 and some other architecture/s.  See http://www.debian.org/mirror/ftpmirror  http://cvs.debian.org/*checkout*/webwml/english/mirror/Mirrors.masterlist?root=webwml  http://bugs.debian.org/  master:~debian/archive/debian-mirrors/  http://mirror.debian.org/status.html (official) http://www.de.debian.org/dmc/today/ (later version of the same codebase)  http://www.apache.org/mirrors/  On the same basis as , currently about 6.9% of ftp.debian.org users look for oldstable. Of those, all but 6.85% appear to do so with apt.
Description: Digital signature