[gopher] Re: New Gopher Wayback Machine Bot

[Benn Newman <newmanbe@sdf.lonestar.org>]

On Wed, Oct 12, 2005 at 04:45:56PM -0700, Cameron Kaiser wrote:
> I'm a little concerned about your project since I do host a number of large
> subparts which are actually proxied services, and I think even a gentle bot
> going methodically through them would not be pleasant for the other side
> (especially if you mean to regularly update your snapshot).
I agree with Cameron and think that until it supports robots.txt, it shoudl be disabled.
I also think it would be a good idea to have it from a single, publically known
domain name.
-- 
Benn Newman
newmanbe@sdf.lonestar.org | gopher://igneous-rock.homeunix.net

Bradley's Bromide:                                                              
        If computers get too powerful, we can organize them into a              
committee -- that will do them in.