[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

[gopher] Re: New Gopher Wayback Machine Bot



On Wed, Oct 12, 2005 at 09:07:52PM -0500, Benn Newman wrote:
> I agree with Cameron and think that until it supports robots.txt, it shoudl be disabled.
> I also think it would be a good idea to have it from a single, publically known
> domain name.

OK.  It is not running, supporting robots.txt, and using both the unescaped
and the escaped selector name.

The IP is that of quux.org, but unfortunately the reverse resolution I
have no control over, so it will look like a DSL address.

I will be doing more work tomorrow to make it be more round-robin so
as to not put much load on any one server at once.

Thanks,

-- John



Reply to: