Re: Mirroring ftp.debian.org
-----BEGIN PGP SIGNED MESSAGE-----
For those using mirror or contemplating it...
You need perl4 for version 2.3 of mirror. The experimental version of mirror
(version 2.8) is supposed to work under many versions of perl including the
latest perl 5. It can be found at
<ftp://sunsite.doc.ic.ac.uk/packages/mirror/experimental>
There is an ultra low volume mailing list for those interested in following
any development or bugs, <mirror-people@doc.ic.ac.uk>. To subscribe send mail
to <majordomo@doc.ic.ac.uk> with a message body containing:
subscribe mirror-people
If you are using mirror-2.3 (this is what I am currently using) then you will
probably want to apply the following patch to the ftp.pl perl package. The
patch (which is fuller and more complete in version 2.8) works around a memory
leak in perl. Without the patch you need the same amount of virtual memory as
the amount of data you wish to transfer :-). This patch is not a complete
fix! It could cause mirror to hang if a remote site stops sending data for
some reason.
*** ftp.pl.~1~ Sun Apr 24 17:36:33 1994
- --- ftp.pl Sun Apr 24 18:04:45 1994
***************
*** 595,601 ****
local( $bytes, $lasthash, $hashes ) = (0, 0, 0);
local( $old_sig ) = $SIG{ 'ALRM' };
$SIG{ 'ALRM' } = "ftp\'read_alarm";
! while( ($len = &ftp'read()) > 0 ){
$bytes += $len;
if( $strip_cr ){
$ftp'buf =~ s/\r//g;
- --- 595,601 ----
local( $bytes, $lasthash, $hashes ) = (0, 0, 0);
local( $old_sig ) = $SIG{ 'ALRM' };
$SIG{ 'ALRM' } = "ftp\'read_alarm";
! while( ($len = sysread( NS, $ftpbuf, $ftpbufsize)) > 0 ){
$bytes += $len;
if( $strip_cr ){
$ftp'buf =~ s/\r//g;
You should be able to use the following mirror script to retreive the whole
archive. I don't normally have the `mirror.defaults' file installed as it can
cause some unexpected things to happen for the novice.
package=defaults
remote_user=anonymous
#Convention says use: your_username_here-mirror@
# remote_password=username-mirror@
do_deletes=true
max_delete_files=100%
max_delete_dirs=100%
timeout=300
retry_call=10
get_patt=.*
package=debian
comment=Debian from ftp.debian.org
site=ftp.debian.org
local_dir=debian
remote_dir=/debian
recursive=true
make_bad_symlinks=true
#You can mail the results of mirror to a user...
# mail_to=
I normally run mirror interactively. If you use the -d -d option then it will
give you a running commentary, including info on the recent transfer rate.
mirror -d -d mirror-debian
Simon Cooper,
Systems Coordinator,
Rutgers University, Network Services.
==============================================================================
>Date: Mon, 2 Oct 1995 12:00:18 -0600 (GMT-0600)
>From: Marcelo Magallon <mmagallo@simula.efis.ucr.ac.cr>
>
>We (a bunch of guys @ Physics' Department - University of Costa Rica) are
>trying to mirror ftp.debian.org, in order to provide a local (Central
>America mostly) mirror for debian. We are using the Perl-based Mirror
>program by Lee McLoughlin <lmjm@doc.ic.ac.uk>, version 2.3. This is the
>first mirror we are trying to run, and we are having some trouble with it.
>We've tested it using local (small) ftp servers, and everything seems to
>be OK, but when try to mirror ftp.debian.org, it's seems like the script
>can't handle it, because it runs out of memory (or at least Perl says so),
>and it quits, leaving the mirror unfinished. When we restart the script it
>"gets confused" and retrieves files already retrieved (actually, SOME of
>the files have changed, but not ALL of them). Bottom line is we haven't
>been able to mirror debian beyond the first 30-40 MB.
>
>We are running Perl 4.0PL36, the one which comes with Slackware 2.0.0,
>which in turn is going to be dumped in favor of Debian 0.93R6.
>
>Any ideas on this subject? Any mirror site`s manager listening? Help...
>please. (My apologies is this is way off-topic)
>
>TIA,
>
>-Marcelo <mmagallo@simula.efis.ucr.ac.cr>
-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
iQCVAwUBMHDX1Xr3R7r4LOqRAQFadwQAltA+TiudqfoP9Mi0nP8eG5RtqVohBq7X
h31EdE3HGI340JIrAtkfzBO7QqPOC5HGMfQjIE5pAxLUZ+M6SjShGG0Q83nzG0r0
26ZjVdziquudZ8eBWYYatVIH2iOYWdQgyG6AAXQRqVuTF14zPmVnqvzmCkVl2J0l
RKBkctMETOg=
=54/l
-----END PGP SIGNATURE-----
Reply to: