[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [gopher] [GOGOPH] Starting crawler



On Wed, 2 May 2012, Damien Carol wrote:

Hello gopherspace,

I will soon start crawling of gopherspace with my new crawler.
I want to start it every weeks at sunday.
The crawler will not send request more than 1 per second.

Please say me if that's correct for you all.
The crawler only get Menu / Text nodes currently. I don't think it will be
too much overload.

I'm waiting your feedbacks.

Regards,
Damien CAROL

ps: you can already test the search engine at :
gopher://dams.zapto.org/1/

Could you make it such that typing "gopher dams.zapto.org" at a Unix command line will bring up your page?

--
David Griffith
dave@661.org

_______________________________________________
Gopher-Project mailing list
Gopher-Project@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/gopher-project




Reply to: