Re: JTE (Jigdo Template Export) v1.0
On Mon, Jun 07, 2004 at 02:02:24PM +0100, Steve McIntyre wrote:
> 1. Modify jigdo so it knows about the internals of ISO images and can
> efficiently scan them (bad, not very generic for jigdo)
>
> 2. Write a helper tool to dump extra information for jigdo to use
> alongside the ISO image (helper tool written, but modifying jigdo
> to use this looks HARD)
>
> 3. Patch mkisofs to write .jigdo and .template files alongside the
> ISO image
>
> I've now done #3, and the patch for mkisofs is at
That's really cool!!! Many thanks for doing this!
Something like #2 was my favourite: Write a special .iso image, but omit
the file data from it, just embed information about the names of files
which were omitted. Later, tell jigdo-file to process the results and write
out proper .jigdo/.template files.
I guess it's my fault for not offering to add support to jigdo-file for
this, sorry Steve. IMHO, it would have prevented some code from being
duplicated in jte and jigdo-file, permitted the use of jigdo-file's cache
for checksums (=> even faster!), and resulted in a smaller patch to mkisofs
which has better chances of being accepted by mkisofs upstream. And
(unsurprisingly) it doesn't look /that/ hard to me. ;)
> To use this code, specify the location of the output .jigdo, .template
> and .jte files alongside the ISO image. The .jte file is an
> intermediate helper file that I'll probably lose for the next
> release.
What does the .jte file contain now, and why is it needed?
> How fast is it?
> ===============
>
> On my *laptop* (600MHz P3, slow laptop disk) I can make a template file
> in parallel with the ISO image from a typical 500MB data set in about 2
> minutes. By simply not creating the ISO (-o /dev/null), this time halves
> again. The data set I'm using here is a copy of the woody i386 r2 update
> CD, as it's a handy image I had lying around.
Yum. :)
> More features:
>
> 2. Add support for -jigdo-exclude option(s), so that we can exclude
> (from the jigdo) README.* etc and other files that go on Debian CDs
> but often change on the mirrors. Reasonably easy to do, and I'm
> playing with this now.
>
> 3. Add pattern-matching in the .jigdo file (e.g. /mirror/debian ->
> Debian:). Again, should be easy.
We'd get both 2. and 3. for free with the "post-processing by jigdo-file"
approach, hmm...
> 5. MUCH harder: re-reading and re-encoding .iso images that have been
> modified since they were first written. This is necessary for
> the boot code used on several architectures in debian-cd. I see how
> to do it - basically diff the image on disk to the one we would
> recreate from the .template file and write a new template file to
> match that. It's going to take some work...
But scanning the changed image would also need a lot of time... I think a
different approach looks more promising: Write a special "dd-jigdo" program
which behaves and works like the original "dd" in every respect, except
that it works directly on the .template. A simple version of such a program
could get away without modifying the .jigdo file, and the .template format
does not make it necessary to recompress all the data in the template -
just a part of it needs to be recompressed.
Alternatively: While you're busy hacking on mkisofs, why not add the
functionality there? The use of dd by debian-cd only illustrates that
mkisofs lacks a feature here!
I think dd is the only tool used by debian-cd, to write a boot sector for
some arches - or is there something else?
Cheers,
Richard
--
__ _
|_) /| Richard Atterer | GnuPG key:
| \/¯| http://atterer.net | 0x888354F7
¯ '` ¯
Reply to: