Re: [gopher] Spidering teh gopherspace
> Anyway, on to a few questions: What kind of spidering rate would the
> admins here accept? The spider will index types 0 and 1 (text documents
> and menus) and currently does three hits per second (actually, a hit and
> a 1/3 second delay). I think that's too fast - so how does a hit per
> second sound like? I'll take forever to spider things, but at least it
> wouldn't kill anyones server....
V-2 uses five seconds, actually, and it only grabs menus (in keeping with
the original Veronica concept), so it is a pretty low impact spider.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser@floodgap.com
-- Artificial intelligence is no match for natural idiocy. --------------------
_______________________________________________
Gopher-Project mailing list
Gopher-Project@lists.alioth.debian.org
http://lists.alioth.debian.org/mailman/listinfo/gopher-project
Reply to: