Re: Creating jigdo files without loal file access
Richard Atterer wrote:
Hi Patrick,
On Mon, Mar 03, 2003 at 12:37:06PM +0100, Patrick Strasser wrote:
In short, I want to produce jigdo files for the Debian/Hurd CD set, but I
don't have access to, don't want to setup , and don't have the space for
a mirror.
My first reaction: This will be difficult! :)
I know :->>>
warning: I've wote this mail during two hours of recherche, so some
parts may sound strange, but I didn't remove thema as they contain
information needed to understand things.
Also note that a problem with .jigdo files for Debian is that the mirror
contents change. As soon as a file on the CD is no longer on the mirror,
jigdo will fail. For this reason, it is mandatory with Debian CDs to set up
a "fallback mirror",
I'm aware of this effect. I have talked to Attila Nagy from fsn.hu,
which host a fallback directory for the snapshots. He said setting up a
fallback directory for Hurd packages would be no problem for two weeks
buffering. I don't expect too much packages to change. As a second way I
could set up a fallback directory by myself after some convincing our
admin to leve me some space for a Good Reason. ;-)
That's right. :-/ One possible trick is to re-use the [Parts] section of
existing .jigdo files. For example, if you were to release a custom version
of an official CD, you could copy the section from the official .jigdo
files (or even more comfortable, use "jigdo-file make-template --merge").
I would be the first to produce Hurd jigdo file sets. So this is no way
to go.
- Obtain list of md5sums and paths on the server
- Obtain list of md5sums needed by the .template of the loop-mounted image:
"jigdo-file ls -t foo.template --hex | grep need-file" (4th column)
Ok, I think I've exactly found what I'm looking for:
Debian has a file called md5sums.gz on it's mirrors, about 5.6 Mb in
size, which is exactly the output ot md5sum, which can be easily be run
local. I'll just match the md5sums and thus will get all the files which
are identical on the mirrors and on the mounted imgage, even with server
path. Then I play arround with the pathes and hopefully get a nice jigdo
file set (sometimes, somehow).
- Find paths for the checksums (e.g. using the "join" utility)
IMHO you really need a little help from someone with
access to the server.
Of course I can talk to people having access, but we additionaly a problem:
Philip Charles, the creator of the Debian/Hurd CDs, is located in New
Zealand, and has only low bandwith capacity. He manages to fetch all
necessary packages, but uploading 5 CDs takes much to long on his
connection. So he gives the images to a friend, who has better
connectivity and loads them up. This is quite complicated and takes
time. That's why I'm investigating a way to build jigdo file sets
without a big local filesystem, which is expensive to keep up to date;
Philip would then build his CDs, make jigdo fiel sets, which are much
smaller, and hopefully load tehm up himself, aving a lot of time. At the
initial upload server someone runs jigdo and blows them up to full size.
So all the CD creator does is managing the meta-information (CD
structure and special packages/files). Thus he does not move around
data, but just the additional data for oraganisation. I like this idea :-)
Greatings
Patrick
--
Engineers motto: cheap, good, fast: choose any two
Patrick Strasser <pstrasser at sbox dot tugraz dot at>
Student of Telematik, Techn. University Graz, Austria
Reply to: