On Tue, 13 Sep 2011, Jeff W wrote:
Anyone else seeing unusually large numbers of hits on their servers today? Normally I get like a dozen or so; yesterday: 491, today:160. Most are from various eastern european countries or indirect via what looks to be some sort of WWW-gopher gateway at bytemark7.spotonnet.co.uk.From the log it looks like intentional hammering to try to crash theserver (it didn't). Jeff
I think it migth be baidubot, its quiet aggressiv in its crawling this days, it crawlers my http sites every day, with 30-40sec between access. But google and bing are also crawling through the gopherproxy.org at bytemark7.spotonnet.co.uk.
As we cant ratelimit, or any other robots.txt things when those web crawlers are hitting us through those proxies, I have opped for simply redirecting access from those proxies to another selector.
is in the TELEFISK_FILTER ifdef in this patch gopher://telefisk.org/0/gopher/gophernicus-1.2-beta-telefisk.diff Regards -- Jacob Dahl Pind | telefisk.org | fidonet 2:230/38.8 _______________________________________________ Gopher-Project mailing list Gopher-Project@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/gopher-project