Re: [gopher] New Gopher Proxy
> > > > In a google search for "gopher proxy" I happened to stumble across this:
> > > > http://gopher-proxy.stephenmorley.org/.
> > >
> > > This is great.
> > > Now if we can just get some of the http-based gopher sites to start
> > > listing these proxies, they might get picked up by the search engines.
> >
> > ... as long as the search engines don't start indexing them *through* the
> > proxies. I've had services get DOSed by bot traffic, and I've had to block
> > proxies that did (I'm not naming names; that was handled under the table).
>
> It seems to me that this would be a good thing, and that
> responsibility for limiting bots on sensitive sites would be the
> domain of the site operator, not the proxy operator.
I completely disagree. Proxies shouldn't be indexable. Just like open mail
proxies foster abuse on mail servers, improperly managed gopher and HTTP
proxies foster the same abuse on the servers they target, inadvertently or
otherwise. More to the point, bots that don't understand what gopher is will
simply crawl over and over, amplifying traffic. This is why I have blocked
misconfigured proxies in the past, and will do in the future if I detect them.
For example, the Twitpher script got DOSed multiple times by a proxy I won't
mention due to HTTP spiders running over and over it, requiring me blocking
the proxy to restore service (leading to a nasty exchange with the proxy
owner) as the proxy exceeded Twitter's anonymous rate limit, which was not
fixed until I requested special dispensation from Twitter to make more
requests. If those had been legitimate users, that would have been one thing,
but these were mindless, automatic bot queries. That's a waste of resources
and cycles.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser@floodgap.com
-- And if I claim to be a wise man/it surely means that I don't know. -- Kansas
_______________________________________________
Gopher-Project mailing list
Gopher-Project@lists.alioth.debian.org
http://lists.alioth.debian.org/mailman/listinfo/gopher-project
Reply to: