Re: md5 sum mismatches and mirror syncs

[Goswin von Brederlow <brederlo@informatik.uni-tuebingen.de>]

martin f krafft <madduck@debian.org> writes:

> also sprach Goswin von Brederlow <brederlo@informatik.uni-tuebingen.de> [2006.08.11.2326 +0100]:
>> The algorithm used is 2pass. First pass only mirrors pool while the
>> second pass mirrors the Release and Packages files. The time a mirror
>> is out of sync should always be limited to the time it takes to
>> download the Packages file after the Release file. All the index files
>> are done in one chunk file to file. A matter of minutes at most.
>
> Or almost 10 hours as was the case today.
>
> So if rsync dies, maybe this is the problem to fix? Is it?

I suspect as much.

Even if you just rsync the whole thing then all files and directories
are processed in alphanumerical order. That means that first all of
debian/dists is synced (Release and Packages files). With a single
pass you get lots of "File not found" errors while the sync is running
because debs (pool) are synced after the index file. But hitting the
right time when Release is synced but Packages not is somewhat
unlikely. Nothing that would remain for 10h.

I only see 3 possibilities of a longer mirror corruption:

1. rsync died

2. rsync made an error and corrupted the file
   Has anyone seen that ever?

3. the sync was started while upstream was out-of-sync
   With push mirroring that means upstream has the problem and
   ultimately some upstream had 1 or 2 happening.


I've seen 1 happening but not 2 personaly.

MfG
        Goswin