Re: [gopher] salsa.net hung connections
Ray Lopez <rl@well.com> wrote:
> Thanks, Cameron. I've been unable to participate in any discussions on
> the mail list and it would be good to be able to do so.
>
> I agree with you that our proxies should NOT be indexable by web
> spiders. It's just too much work, and goes against the basic philosophy
> of gopher vs. web.
>
> I've implemented changes on my server to hopefully alleviate some of the
> traffic and will be working with James Mills to make the necessary
> changes to his gopherproxy software.
>
> RL
Ray - thanks for the prompt response. And +1 on not wanting web spiders
crawling gopherspace via the proxies. I don't think there is much I could
do on my end; maybe traffic throttling or temporarily dropping connections
when it looks like spider activity. I don't have much faith in crawlers
minding the robots.txt entries.
Jeff
_______________________________________________
Gopher-Project mailing list
Gopher-Project@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/gopher-project
Reply to: