cpu usage, consideration of others, apt-listbugs

[Adam Heath <doogie@brainfood.com>]

I was looking at the various stats programs(http://master/mrtg/), and noticed
that for the last week, master's incoming bw, outgoing bw, and load, have been
unusually high.

After some digging, I found 2 main causes.

1: The apt-listbugs author has seemed it nescessary to export all debbugs
   data, by way of a symlink, and then to allow admins to request bug reports
   when packages get updated.  This was not done with any consultation with
   any BTS admin, nor any admin that is invovled in the machine that would be
   handling this increased load.

   A quick grep thru the access.log for the bts, shows 116k total lines, and
   48k lines are for apt-listbugs, requesting .status files, or the index
   files.

   What is not shown, is the requests for the actual bug reports.  These can't
   be easily tracked, as apt-listbugs uses querybts(from reportbug), and a few
   other fallbacks, so it's not easy to match up those lines.

   As such, I have used mod-rewrite to capture all requests that contain
   'apt-listbugs' in the User-Agent setting, and redirected them to a
   placeholder page, explaining the issue.

2: unrestricted, unlocked, cpu hogs

   I noticed at least one person on master running a spamassassin in his
   .procmailrc.  There was no lock file, which means multiple instances
   might(and actually did) run at once.  spamd/spamc wasn't being used, so
   there was even increased load.

   The BTS has run a local spamassassin for 6+ months now, but it was done
   correctly from the beginning: a spamd, running as the debbugs user, and a
   spamc that talked to it.  The spamc call was actually done in a shell
   script wrapper, that checked to see if spamd was running, and if not,
   restarted it.

   The BTS also serialized all spamc calls, so at most only one would ever be
   running.

master runs the bugs.debian.org, cgi.debian.org, and lists.debian.org(the web
archive, not the mail software).  The former are cpu intensive.  The latter
sometimes is, when a search is done.

Because of the above usage(not only the web pages, but the backend
scripts(index updaters, etc)), it is not best to run high-demand cpu
applications on that machine.

Please, be considerate of others who use the resources that are donated.
Please don't abuse.