Bug#720970: packages.debian.org: ban check_http hits on 1MB files

[Simon Paillard <spaillard@debian.org>]

Package: www.debian.org
User: www.debian.org@packages.debian.org
Usertags: packages.debian.org
Severity: minor

Running 'visitors' package on packages.d.o logs produced:
http://people.debian.org/~spaillard/packages.debian.org-access.log.html

Especially looking at
http://people.debian.org/~spaillard/packages.debian.org-access.log.html#Requested%20pages

1,7% of requests are against a single 800kB file, and closer look show it's
actually a nagios check from some specific IPs !!

-> They should be blocked IMO.

$ grep allpackages?format=txt.gz packages.debian.org-access.log-20130820 | cut -d '"' -f 6 | sort | uniq -c | so
   6748 check_http/v1.4.15 (nagios-plugins 1.4.15)
     10 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

I've tested the folloing rule, but it doesn't do the trick..
   RewriteCond %{HTTP_USER_AGENT} ^check_http.*
   RewriteRule^/stable/allpackages\?format=txt\.gz - [F]



-- 
Simon Paillard