Re: [KORG] debian-cd mirror out of control
On Tue, 7 Feb 2006, H. Peter Anvin wrote:
Hello,
We are having some serious issues with the Debian mirror on kernel.org.
These need to be resolved, or we will be forced to discontinue it.
Problem #1: sheer size. Currently the full debian-cd mirror occupies
1.3 TB. It shares a filesystem with nothing but debian, and that 1.5 TB
filesystem is about to hit the roof.
Drop anything but the latest stable release, that is everything but the
stuff in: http://cdimage.debian.org/debian-cd/
Problem #2: jigdo files that refer to files that are not present in the
Debian set. Every day, we get references like the following in the Jigdo
logs:
[snip 3.0r2 stuff]
See answer to #1. :)
This wouldn't be such a big deal, if jigdo wasn't such a total CPU hog, and
the above phenomenon means that it runs over and over and over again.
I would appreciate your advice.
At http://www.acc.umu.se/~maswan/debian-push/cdimage/ I have an example
script that looks where the "current" symlink points, then runs
jigdo-mirror on the stuff listed in project/build/$current (currently CDs
for everything, dvds for i386&ppc). The actual script might be "slighlty"
crap, patches are most welcome.
Otherwise a plain rsync on cdimage.d.o::debian-cd should work fine. We
keep that size down to a size that's probably acceptable, even if it is a
bit on the large side. The old stuff is supposed to be at:
http://cdimage.debian.org/cdimage/archive/ - but that isn't active enough
to need mirroring.
The only downside to plain rsync is that you're likely to have to wait a
day or so at release time, if it is a busy release. Preferential treatment
(access to a private rsync module without strict user limits) is given to
those (at least attempting) to do the jigdo-mirror thing.
/Mattias Wadenstein
Reply to: