[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

New mirror scripts for Debian mirrors


Recently, we rewrote the Debian archive mirror script formerly known as 
anonftpsync.  The new scriptset has many more features, and is the new 
de-facto standard for Debian mirrors.

Please switch to it as soon as possible.

The scriptset is available in two forms:
 - as a tarball from http://ftp-master.debian.org/ftpsync.tar.gz
 - as a git repository:
   git clone https://ftp-master.debian.org/git/archvsync.git

If you used anonftpsync in the past, the switch to the new scriptset 
should be fairly easy.

 * Unpack / git checkout it in your mirror user's home directory
 * Change your .ssh/authorized_keys to point to the bin/ftpsync script
   instead of anonftpsync
 * Copy etc/ftpsync.conf.sample to etc/ftpsync.conf and edit it to fit
   your system (you want to look at the TO= and RSYNC_HOST= variables at
 * Test it by manually running bin/ftpsync

The README file supplied with the script should help you and the
sample configuration file is extensively documented.

Debian mirror script minimum requirements
While we don't require you to use our scriptset, we strongly encourage it.
If you do use your own scripts, the minimal functionality that must be 
supported is as follows:

 - MUST perform a 2-stage sync
   The archive mirroring must be done in 2 stages. The first rsync run
   must ignore the index files.  The correct exclude options for the 
   first rsync run are:
  --exclude Packages* --exclude Sources* --exclude Release* --exclude ls-lR*
   The first stage must not delete any files.

   The second stage should then transfer the above excluded files and
   delete files that no longer belong on the mirror.

   Rationale: If archive mirroring is done in a single stage, there will be
   periods of time during which the index files will reference files not
   yet mirrored.

 - MUST not ignore pushes while running.
   If a push is received during a run of the mirror sync, it MUST NOT
   be ignored.  The whole synchronization process must be rerun.

   Rationale: Most implementations of Debian mirror scripts will leave the
   mirror in an inconsistent state in the event of a second push being
   received while the first sync is still running.  It is likely that in
   the near future, the frequency of pushes will increase.

 - SHOULD understand multi-stage pushes.
   The script should parse the arguments it gets via ssh, and if they
   contain a hint to only sync stage1 or stage2, then ONLY those steps 
   SHOULD be performed.

   Rationale: This enables us to coordinate the timing of the first
   and second stage pushes and minimize the time during which the
   archive is desynchronized.  This is especially important for mirrors
   that are involved in a round robin or GeoDNS setup.

   The minimum arguments the script has to understand are:
   sync:stage1   Only sync stage1
   sync:stage2   Only sync stage2
   sync:all      Do everything. Default if none of stage1/2 are
   There are more possible arguments, for a complete list see the
   ftpsync script in our git repository.

The Debian website /mirror will be updated soon.

bye, Joerg
<andreasj> Also diese neuen Spam-Mails muten an wie Blog-Posts von Clint Adams
<andreasj> irgendwie ist es eine Geschichte, aber ich versteh sie nicht

Attachment: pgpJtDSkjtgEC.pgp
Description: PGP signature

Reply to: