[gopher] Re: New Gopher Wayback Machine Bot
On Wed, Oct 12, 2005 at 09:07:52PM -0500, Benn Newman wrote:
> I agree with Cameron and think that until it supports robots.txt, it shoudl be disabled.
> I also think it would be a good idea to have it from a single, publically known
> domain name.
OK. It is not running, supporting robots.txt, and using both the unescaped
and the escaped selector name.
The IP is that of quux.org, but unfortunately the reverse resolution I
have no control over, so it will look like a DSL address.
I will be doing more work tomorrow to make it be more round-robin so
as to not put much load on any one server at once.