Re: Enlighten me about jigdo ..
First, I'm glad the response is from You, the jigdo author .. right ?
On Friday 14 November 2003 12:41, Richard Atterer wrote:
> > 1. the templates are BIG - as I see from 5 to 20 MB for woody
> > (600+ MBs for i386 sid, but I suppose the maintainer of those unofficial
> > templates has messed the things up, since this files are as big as an
> > iso) I wonder what this binary data inside them is. What do they need
> > except relative file path, e.g.
> > pool/main/a/aalib/aalib-bin_1.4p5-18_i386.deb and possibly an md5sum for
> > each file ..
>
> True, the templates shouldn't be so big, this is a result of 1) an "exclude
> list" used by the creation scripts of files which change frequently on the
> server, causing their data to end up in the template,
Which file is this, I thought the debian pool is just for that, being
able to reproduce exact state of several distributions back. Otherwise
that pool might contain just the files for the latest and greatest stuff.
> > Ignoring rsync precence(pipelining - no need to establish TCP connection
> > for each file, repair of broken downloads .. ) is a really bad idea
>
> An rsync-based scheme (whose design inspired that of jigdo) was used in the
> old days, the "pseudo image kit" (PIK). The use of rsync turned out to be
> problematic (too much load on the server), so one of the aims of jigdo was
> to get rid of rsync. All in all, the PIK worked well, but it didn't scale
> to large numbers of parallel downloads.
This is a big step backward. Debian has 300 mirrors and push mirroring
already running. Loadbalancing should not be problematic in that case.
If a particular server gets too busy it might limit the number of concurrent
connections. Priority connections might also be considered(e.g. always
allow connection from other mirrors while keep the regular downloaders number
in certain amount). And the cpu power certainly gets cheaper faster than the
bandwith ..
Something more. It's proven to work. Look at the rsync servers at let's say
mirrors.kernel.org or planetmirror.com.
>
> BTW, one mustn't underestimate the fact that any rsync-based scheme
> requires the Debian mirror admins to install rsync servers. Furthermore,
> the complete images must be stored on several servers somewhere...
This ofcourse is not true. Note that I'm saying using the debmirror
for keeping the files uptodate. Although I'm recommending using rsync
it supports http also.
> Furthermore,the complete images must be stored on several servers somewhere...
Why is that ? You make the images with mkisofs, the pool, and the file list,
remember ?
> > 3.use mkisofs to burn your image. Note I currently can't find a way to
> > achieve this last step. What is necessary here is to have a file with the
> > layout of the debian cds to be supplied to mkisofs(or as an input to a
> > script which creates the necessary directory structure) and an isoimage
> > for the bootable cd. Probably some other minor details as cdlabels .. The
> > cdlayout file is an ascii text file ofcourse and could also be rsynced.
> > The jigdo-file(1) says you need the iso image to create the .template and
> > .jigdo files, so I suppose the mentioned layout is actually used by the
> > people making the iso-s ..
>
> I faced the same problem as you in 3. - it is an absolute requirement for
> Debian CDs that the users end up with a bit-exact copy of the original
> image, so IMO creating it with mkisofs is not an option. :-/
>
First are You sure the exact reconstruction of the iso is not possible with
mkisofs ? What are your concerns, different versions of mkisofs producing
different isoimages (e.g. this iso made with mkisofs-20031114cvs in the
image itself) ? Have You consulted the source or contacted the mkisofs
about that ?
Second, if first could not be achieved. Why is it an "absolute requirement" ?
I personally don't mind isos with differend md5s if the files inside it
might be verified and proved authentic. I see no problem in adding
an option(if it doesn't exists) in the debian installer "verify all
installed packages with respect to CDROOT/md5sums". There might be
an option also to verify CDROOT/md5sums over internet if a connection is
available. Otherwise the user might do that later by running rsync for
that particular file, and transfering abount let's say 20kb.
Note this is only necessary for persons obtaining those images and
not making them with mkisofs. If you keep your mirror uptodate over rsync
and running mkisofs personnaly the md5s of the isos will not be a concern.
Regards
Delian Krustev
Reply to: