[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Outdated woody-i386.list file


On Tue, Sep 04, 2001 at 01:01:17PM +0100, Simon Wood wrote:
> I was typing up a long, rambling email about P-I-K but I'll
> spare you all and just post the conclusions that I came to.

I got equally fed up with the PIK last December, and have been working
on a replacement since then. Maybe you've seen my recent message about
Jigsaw Download, or "jigdo".

> 1). I think that the '.list' and ISO's should contain either
> a revision number (for Official releases) or a date stamp (for
> the testing/unstable 'releases'). This would prevent confusion
> as to which version they and what they contain.

The final ISO images already contain this AFAIK. The .list file isn't
really sophisticated enough. The equivalent .jigdo file, however, is
planned to support "Description" fields in which version info etc. can
be listed.

> 2). There are a number of files that not revisioned controlled
> (mainly documentation and the 'current' boot disks). At present it
> looks like P-I-K is relying on the rsync severs to ensure the latest
> revision. I may be desirable to remove the load on these machines.

Indeed - rsync is IMHO just not suited for a "one server, many
clients" situation. jigdo eliminates the use of rsync completely - all
the data that isn't part of any files on the FTP server, such as
unpacked documentation, ends up in what I call the ".template" file
for the image.

> 3). It would be nice to be able to specify two (or more) sources for
> P-I-K so that the minimum of downloading can be done. For example
> the ISO for Woody (Dec2000) contains a number of packages which have
> not changed - if I have that CD then I should be able to make use of
> it and not have to download them again from a mirror.

This was an explicit design goal of jigdo. The latest prerelease
(0.5.2) already supports this.

What it doesn't yet do is support the downloads, although it's quite
easy to write a script that uses wget for actually fetching the data.
Bear with me, I'm working on it...

> 4). There seems to me that there is a problem with using 'current'
> on the CD's. On the ftp servers this is a sym-link but on the ISO it
> appears just to be a duplicate set of data. Can someone please
> confirm this?

I haven't downloaded any ISOs recently - on my old potato CD,
"dists/potato/main/disks-i386/current" is a symlink to another name in
the same directory. But note that somehow mkisofs seems to be able to
create "hard directory links"; maybe the data isn't actually

> 5). The package mirrors only store the most recent version of the
> packages. This is what is making the generation of a 'outdated' CD
> difficult. It may be desirable to have an archive somewhere of all
> versions of packages, or maybe just the ones from the most recent
> ISO image.

True - and in my eyes this is going to be the biggest problem with
jigdo, which /cannot/ use rsync and thus will *fail* to generate the
image if some older files are no longer present.

I have been thinking about a fallback mechanism like this: jigdo
allows you to specify alternative download locations. If the file is
not found in any of the primary locations, a secondary query is made. 
The server(s) listed as locations for this secondary query then have a
similar function as the rsync servers have now - they should only be
used if the files cannot be retrieved from the normal mirrors.

Also, I'm afraid testing and unstable move too fast for jigdo to work,
even if new jigdo CD images were generated weekly - to distribute CD
images of testing, dinstall will probably have to be modified not to
delete old .debs too quickly. :-/

> 6). The difficult in creating ISO images can only hinder the testing
> of them. This will have a knock on effect to people who have limited
> or no internet connection.

Definitely. So far, I have had to recommend to friends to get the
complete ISOs somewhere else, and not to use the PIK. The PIK works
so-so under Linux - under Solaris or even Windows I've had lots of

> The long and short of it all is that I shall grab the raw ISO
> tonight to get me going..... but I would like to assist in ensuring
> that I won't have to do it again in the future.

You are welcome! Are you a programmer? :-)

Currently, it would be great if someone could set up a beta jigdo
server which generates ~weekly images. However, this requires shell
access on a Debian mirror, lots of CPU cycles on that machine, and the
ability to set up user cron jobs. Additionally, you need to know shell
scripting to knock up a basic replacement for the PIK that uses

See <http://atterer.net/debian/> for more info on jigdo.



  __   _
  |_) /|  Richard Atterer     |  CS student at the Technische  |  GnuPG key:
  | \/¯|  http://atterer.net  |  Universität München, Germany  |  0x888354F7
  ¯ ´` ¯

Attachment: pgp7wPuXBPihB.pgp
Description: PGP signature

Reply to: