Re: [gopher] Spidering teh gopherspace
Kim Holviala wrote:
> Yeah, me again.
>
> Since I got the server done I promised to start writing clients. But
> instead, I got sidetracked and decided to code a search engine first...
>
> I got a basic spider up and running (pure POSIX C again) in a couple of
> hours and succesfully tested it against my own server. Which then first
> throttled me (1-second delay before reply), and after I cleared the
> sessions inetd decided that it had enough and kicked me totally out.
You may be interested in
http://git.complete.org/z-old/gopherbot/
though my goal was somewhat different; I aimed to archive all of
gopherspace, not search it. I believe it was actually successful; I
burned some DVDs and shared them with a few people on this mailing list
at the time. IIRC it was about 5 DVDs.
-- John
_______________________________________________
Gopher-Project mailing list
Gopher-Project@lists.alioth.debian.org
http://lists.alioth.debian.org/mailman/listinfo/gopher-project
Reply to: