Re: New mirror scripts for Debian mirrors
are you sure that the "set -u" should be used? I get a lot of "VAR:
unbound variable" messages and ftpsync aborts.
Joerg Jaspert wrote:
Recently, we rewrote the Debian archive mirror script formerly known as
anonftpsync. The new scriptset has many more features, and is the new
de-facto standard for Debian mirrors.
Please switch to it as soon as possible.
The scriptset is available in two forms:
- as a tarball from http://ftp-master.debian.org/ftpsync.tar.gz
- as a git repository:
git clone https://ftp-master.debian.org/git/archvsync.git
If you used anonftpsync in the past, the switch to the new scriptset
should be fairly easy.
* Unpack / git checkout it in your mirror user's home directory
* Change your .ssh/authorized_keys to point to the bin/ftpsync script
instead of anonftpsync
* Copy etc/ftpsync.conf.sample to etc/ftpsync.conf and edit it to fit
your system (you want to look at the TO= and RSYNC_HOST= variables at
* Test it by manually running bin/ftpsync
The README file supplied with the script should help you and the
sample configuration file is extensively documented.
Debian mirror script minimum requirements
While we don't require you to use our scriptset, we strongly encourage it.
If you do use your own scripts, the minimal functionality that must be
supported is as follows:
- MUST perform a 2-stage sync
The archive mirroring must be done in 2 stages. The first rsync run
must ignore the index files. The correct exclude options for the
first rsync run are:
--exclude Packages* --exclude Sources* --exclude Release* --exclude ls-lR*
The first stage must not delete any files.
The second stage should then transfer the above excluded files and
delete files that no longer belong on the mirror.
Rationale: If archive mirroring is done in a single stage, there will be
periods of time during which the index files will reference files not
- MUST not ignore pushes while running.
If a push is received during a run of the mirror sync, it MUST NOT
be ignored. The whole synchronization process must be rerun.
Rationale: Most implementations of Debian mirror scripts will leave the
mirror in an inconsistent state in the event of a second push being
received while the first sync is still running. It is likely that in
the near future, the frequency of pushes will increase.
- SHOULD understand multi-stage pushes.
The script should parse the arguments it gets via ssh, and if they
contain a hint to only sync stage1 or stage2, then ONLY those steps
SHOULD be performed.
Rationale: This enables us to coordinate the timing of the first
and second stage pushes and minimize the time during which the
archive is desynchronized. This is especially important for mirrors
that are involved in a round robin or GeoDNS setup.
The minimum arguments the script has to understand are:
sync:stage1 Only sync stage1
sync:stage2 Only sync stage2
sync:all Do everything. Default if none of stage1/2 are
There are more possible arguments, for a complete list see the
ftpsync script in our git repository.
The Debian website /mirror will be updated soon.
Panagiotis J. Christias Network Management Center
email@example.com National Technical Univ. of Athens, GREECE