On Mon, Jan 03, 2005 at 08:50:28PM +0100, Richard Atterer wrote:
>On Mon, Jan 03, 2005 at 05:32:58PM +0000, Steve McIntyre wrote:
>> Several of the jigdo files I'm creating end up using the following file:
>>
>> LV13Ru06DGf3YL3ExzFEfw=Debian:pool/main/l/lablgl/liblablgl-ocaml_1.01-2_mips.deb
>
>That checksum is incorrect! :-/ This .deb has a checksum of
>QYrvxbZD6hsrbVCjL1sAWQ on all the mirrors I've checked.
>
>The reason for that problem is probably an inconsistent jigdo cache file. Run
>
> jigdo-file --cache ~/jigdo-cache.db md5 /mymirror/pool/main/l/lablgl/liblablgl-ocaml_1.01-2_mips.deb
>
>to verify this. If the wrong sum ("LV...") is printed, touch the file and
>repeat the command.
>
>Of course, the "better safe than sorry" approach would be to delete
>~/jigdo-cache.db, but it'll take a while to rebuild... :-/
My mirror _and_ the jigdo-cache.db both give the right answer
(QYrvxbZD6hsrbVCjL1sAWQ), which is the really confusing part.
>I've downloaded your template file and at first sight "jigdo-file ls -t
>woody-powerpc-1.template" shows some _really_ weird things: Our LV checksum
>appears in numerous places all over the image.
>
>[...some intelligent guesses later:]
>That LV checksum is the md5 checksum for 53818 bytes of zeroes. :-((
OK, that would explain _something_.
>That explains a few things - wherever there are longer stretches of zeroes
>in the image, references to the zero file are repeated many times.
>
>More importantly, if such an entry made its way into the jigdo cache, this
>means that jigdo-file will slow to an absolute crawl for any images which
>contain larger stretches of zeroes - such as the DVD images!! :-( This
>might explain why you haven't been quite that content with jigdo-file's
>performance of late!
>
>So how did the all-0 entry end up in jigdo-file's cache? I have no idea,
>and this looks like one of those horrible bugs which are impossible to
>find. :-/
Quite.
>One possible reason which would allow me to "blame someone else" ;-) would
>be that jigdo-file scanned this file for the first time when it was just
>being mirrored. Does rsync (or whatever mirror method you use) first set
>the extent and timestamp of a file and then transfer the data? Hmm, that
>sounds unlikely - any suggestions on how I could track this down?
No, rsync is quite clever. It grabs new files down as temporary file
names and then renames them into place when they're finished.
>> I've used JTE to generate the other jigdo files, and it doesn't
>> exhibit this problem thankfully...
>
>If you use it for the release images, this will also be a nice way of
>testing it and ironing out any remaining bugs - IMHO, this is good since
>you said you want to use it for the sarge release images!
It's being used now by Manty. There's a small issue with m68k at the
moment, but otherwise it seems to work just fine. There are problems
using it with woody for some of the architectures where the boot stuff
has changed for sarge and re-writing it for a couple of woody point
releases is just too much effort. Hence I'm using the original scripts
and jigdo-file for the awkward arches (mipsel, ppc and sparc for now).
>I just wish I could find the cause for this problem with jigdo-file. :-/
If it helps as a clue, this file is _exactly_ the same one that caused
me pain for the woody r3 images. That's why I just commented it out
from the jigdofilelist for the next run. Once I have this run
finished, I can re-run with whatever debug enabled that you'd
like. Hell, if it helps I can give you access to the system here to
play with it yourself.
--
Steve McIntyre, Cambridge, UK. steve@einval.com
"I've only once written 'SQL is my bitch' in a comment. But that code
is in use on a military site..." -- Simon Booth
Attachment:
signature.asc
Description: Digital signature