[gopher] Re: Gopherspace archive
Cameron Kaiser wrote:
>> :)). With all the software and binary stuff taken out, I think it should
>> (nearly) manageable. The index file shouldn't be nearly as big as the
>> whole archive, I could then make a front end to that (yay for sed and awk!).
>
> Looking at the size of V-2's index file, I wouldn't count on that if you have
> any kind of useful indexing. Yes, you won't be indexing binaries, but a large
> text file will have *plenty* of keywords, and an index of keywords itself has
> to be indexed by your database engine to be usefully searchable. That'll
> consume a platter or two.
>
> I haven't decided yet if I can support hosting the archive. This sounds like
> a job for BitTorrent unless the other solution is to unpack the archive and
> let people take the pieces they want instead of downloading the whole thing.
> My uplink is a woeful 608k max ADSL line and the web and gopher servers both
> fight over it on a daily basis.
>
If you would read the paper
(We interrupt this e-mail for an MLA-ish citation!
Lesk, M. E., "Some Applications of Inverted Indexes on the UNIX System."
Murray Hill, New Jersey: A really long time ago
We now return to our e-mail)
you would find out that there is a maximum number of keys per file/entry
(or it is supposed to to anyway, documentation not meeting reality adds
spice to life (or something like that)). I would not be writing the
engine (I don't hate myself *that* much), just a Gopher front-end. I
never said it would be "fast."
I think bittorrent is the way to go.
--
Benn Newman
-- Binary/unsupported file stripped by Ecartis --
-- Type: application/x-pkcs7-signature
-- File: smime.p7s
-- Desc: S/MIME Cryptographic Signature
Reply to: