Re: [gopher] gopher proxies
> > AhrefsBot is relentless. They don't seem to obey robots.txt, and
> > will just hammer a server for non-existent links. So are you
> > suggesting that it's the proxy that should be responsible for
> > filtering this in some way?
>
> Some of them already do. The floodgap proxy, for instance, has a blackhole
> for bots to wander about in.
The proxy doesn't. The webserver does -- I don't want abusive bots on my
sites at all.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser@floodgap.com
-- Don't anthropomorphise computers. They hate that. -- Gregory McNear --------
_______________________________________________
Gopher-Project mailing list
Gopher-Project@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/gopher-project
Reply to: