Re: rsyncing images
On Fri, 15 Sep 2000, Hugo van der Merwe wrote:
> On Fri, Sep 15, 2000 at 12:29:59AM +0200, J.A. Bezemer wrote:
> > I just don't understand... I know by now that rsync does behave strangely
> > sometimes, but I've never heard anything like this. You might try
> > --block-size=20000 or even 123456 the next time, maybe that helps.
> > (And I think starting from the pseudo-image+330M again (if you still have it)
> > would work best.)
>
> I tried the same mirror with 20000, skipped the 330M, then started
> downloading diligently again... I know that they use an older rsync
> though... now trying a mirror using same version of rsync, with 123456.
>
> <time passes>
>
> Interesting: I think it downloaded one block near the beginning...
> (Whereabouts was the bad bit in the second image...?) but it is also
> downloading everything from 330299209... I think I'll try regenerating
> the pseudo image again. And then adding the 440Meg segment at the end...
> (any reason whatsoever to rather add the 330? Even if the extra 110 is
> corrupt, it will then simply be ignored?)
Rsync will just ignore corrupt stuff (that's what it's for ;-), but if you
know it's corrupt beforehand, there's no reason to add it, since it'll only
cost CPU cycles for calculating block-checksums that aren't used at all.
> > Hmm. You were talking about a local mirror; if you're accessing it by FTP it
> > just _might_ be that it decided to switch to serving in ASCII mode halfway
> > making the pseudo-image. The easiest way to check that is to open the image in
> > mc(1)'s viewer (F3), then F4 to switch to hex mode and use F5 to get somewhere
> > `deep' in the offending part. There must be present both `single' 0A's and
> > `single' 0D's. If you see too many OD-OA's, then that's the problem.
>
> There are many sing 0D's, and many single 0A's, and a couple of 0D0A's.
Okay, then that's not your problem.
> $ hexdump binary-i386-2.iso | grep 0d0a | wc
It's 0a0d with hexdump (todos < /etc/passwd | hexdump). od -tx1 -Ax does a
'better' job. Hoewver both 0D-0A and 0A-0D should occur approx.
filesize/65535 times in zipped data.
I just thought of a different strategy for your next pseudo-image. You can
split(1) the .list file in some 10 parts, and make (partial) pseudo-images for
each of them. Then cat them together (+ the already-rsynced part); if things
go wrong again, you'll know approximately which part is corrupt and can
rebuild only that one. But, more interesting, you can easily extract the first
file of each part (head -c) and check if the md5sum matches with the Packages
file. If not, try to compare them at the byte level by od'ing both versions.
Or try to build one (smaller) part twice or more, and see if the md5sum
changes.
Regards,
Anne Bezemer
Reply to: