Re: md5 sum mismatches and mirror syncs
martin f krafft <madduck@debian.org> writes:
> Failed to fetch
> http://ftp.ie.debian.org/debian/dists/sid/main/binary-i386/Packages.gz
> MD5Sum mismatch
>
> I am seeing a lot of this stuff lately, and I've been told it's due
> to mirror syncs. As our archive grows bigger, the sync takes longer,
> so this problem will happen more often in the future.
>
> I wonder why. To me it seems as if a sync is a blind rsync, which
> copies the Release index before the individual Packages/Source
> indices.
>
> Shouldn't we switch to using/advocating a smarter algorithm like
> the one debmirror or anonftpsync use, which is to push new package
> files to the archive, then synchronise indices, then delete obsolete
> package files?
>
> Or is this already in place? Why then do we see errors like the
> above?
The algorithm used is 2pass. First pass only mirrors pool while the
second pass mirrors the Release and Packages files. The time a mirror
is out of sync should always be limited to the time it takes to
download the Packages file after the Release file. All the index files
are done in one chunk file to file. A matter of minutes at most.
For details please read the recommended mirror scripts provided by
debian.
I think the problems seen way too often come from rsync failing every
now and then leaving a partialy updated mirror. I've seen that happen
several times but never reproducible.
MfG
Goswin
PS: All mirror admins and Debians default scripts should add the
"--delay-updates" option from rsync to cut down the out-of-sync time
to near nothing.
Reply to: