On Thu, Dec 04, 2003 at 01:25:15PM +0100, Richard Atterer wrote: >On Wed, Dec 03, 2003 at 11:53:13PM +0000, Steve McIntyre wrote: >> I've been poring over the jigdo source code, and I'm looking at the >> main loop in MkTemplate::scanImage(), especially the code where it >> compares the current hash against the hashes cached for known files. I >> don't know if you've thought of this (you may even have tried it, and >> it runs even slower), but when we first start looking for matches (a >> new file), could we not possibly short cut some of the work here by >> using the directory information that we have from the CD image itself? >[...] > >IMHO all the iso9660-aware code would be difficult to write - see the >following suggestion from my HACKING file, which I think is a better way to >speed things up: > >".rawtemplate" support: > > It is possible to make .jigdo creation for Debian CDs much faster > by hacking mkisofs to support more direct output of .jigdo/.template > files: OK, I now have something approximating this. First-cut patch for current mkisofs is at http://www.einval.com/~steve/sw/CD/mkisofs.diff Look at struct jt_extent_data in mkisofs.h for details of the output format I'm using - it's simply a concatenated set of header n x matched file / unmatched extent footer dumped to the output file specified on the command line using the new -jigdo-template option. I've tested the code here fairly well over the weekend, and I'm quite happy with what I have - it works, and doesn't interfere at all with normal ISO image output. For now I'll only produce the template file if you also make an image; if you don't want the image, simply specify -o /dev/null to throw it away. The internals of mkisofs are really very nice - it's well structured and clean in most places, and it was obvious how and where to make the first attempt at these changes. They'll probably want cleaning up before upstream will accept, of course. I've also written simple dirty tools to list the contents of one of these template files and a current jigdo file - see http://www.einval.com/~steve/sw/CD/dump.c (jigdo template) http://www.einval.com/~steve/sw/CD/dump-jte.c (helper template) I've had a look at adding support for the new template helper file into jigdo itself, but I can't abide C++ at the best of times and the jigdo code is fairly opaque IMHO. One thing I'm curious about, Richard - why dod you decide to use (almost) base64-encoded md5sums in the jigdo files? All other software works with simple hexdumps of md5sums, and I'm not convinced doing things differently is a good idea... Comments? -- Steve McIntyre, Cambridge, UK. steve@einval.com Into the distance, a ribbon of black Stretched to the point of no turning back
Attachment:
pgp1YjdVq4kqs.pgp
Description: PGP signature