[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [gopher] Torrent update

On the Sat, May 01, 2010 at 06:23:09AM +0300, Kim Holviala blubbered:


> PLEASE set up robots.txt to prevent robots from re-archiving this stuff!  
> I don't want to end up with 20 copies of the 30gig archive!
> Put "robots.txt" in your gopher root with something like this in it:
> User-agent: *
> Disallow: /archives

Does this really work for gopherspace? 
Besides, are there any popular search engines that index
gopherspace? Google[1] does not, neither does Bing.

CU, Venty

[1] I once made a joke to a Google engineer about when "Google
Gopher" would launch. He looked kind of puzzled...

Verpassen Sie nicht den neuen Hackerfunk irgendwann im Mai.
Wie immer von 19:00 bis 20:00 Uhr auf Radio LoRa in Zuerich.

http://www.hackerfunk.ch/      http://www.lora.ch/

Gopher-Project mailing list

Reply to: