Suggestion - when jigdo fails. (also: speed improvements)
Hi all,
I have been trying out jigdo-lite to download/build both potato and
woody images. So far, 5 out of 6 went off without a hitch - very cool
process! The one time it failed, I got the "Aargh - not all files could
be downloaded" message. I saw the message that said if this happens, my
only recourse was to revert to rsync.
FORTUNATELY, I had no idea what this meant. (I know what rsync is, but
there's not enough information there to tell me what to rsync from where,
how do I know what files I want, etc.) Anyway, since I was blissfully
ignorant, I started looking for simpler options. I scrolled up and
examined the output from jigdo, where I saw that the ftp server was
complaining that some of the files were not found on the server. (Don't
know why - incomplete mirroring?) So I re-ran jigdo-lite again, but I
specified a different mirror site (for both US and non-US), and the
files were found: the process ran to completion and the ISO checksummed
correctly!
So: I just wanted to share that in case others have similar problems
using jigdo, since -- if it works for you -- it's so much easier
(and faster) than restarting the download or trying to figure out what
you're supposed to be rsyncing.
BTW, whoever maintains the web page at http://www.debian.org/CD/jigdo-cd/
might want to mention this there as well, since I think this is where
most users hear about jigdo for the first time.
Lastly, the one not-completely-cool things I noticed about jigdo is that,
since many of the files are small and download relatively quickly (at
DSL speeds,) the single-ftp-logon-per-file nature of the way wget is
being used means you spend a lot of time connecting, logging in
anonymous, bringing up a tcp connection, etc. In short, my DSL modem is
being used at far less than it's maximum rate, so the process takes
longer than it needs to. I'm kind of kicking around the idea of
modifying it a bit to:
1) ask wget for more than one file at a time, to reduce the ftp login
overhead, and
2) allow the entry/selection of multiple mirrors, so that I can
distribute my load aross multiple mirrors. (I don't mind maxing out
*my* link, but It'd be nice if I could spread that load across a few
mirrors so as not to monopolize one mirror's bandwidth. (As a side
effect, this might also have the effect of reducing the probability
of the "Aargh!" problem, since, if a file failed at one mirror, I would
try another.
Any suggestions on other things I could try to maximize download speeds
and/or improve mirror-friendliness/robustness are welcome.
Cheers,
-(-- John V. Martinez
jvmatl@mindspring.com
Reply to: