Dealing with web bots
I had noticed a rather persistent webbot that kept trying to make web
requests of my gopher server. It wasn't really hurting anything (it wasn't
trying every second for instance) but it was annoying. And they weren't
picking up on the fact that what they were talking to wasn't a web server.
After some thought, I decided to detect these requests (not hard), and
return an actual HTTP response, and for that, I choose 418 "I'm a teapot"
(RFC-2324) on the grounds that it's a client error code, and more
specifically, a *joke* client error code (and I further include the Teapot
song [1] in the responce). I'm glad to say that it cleared up the problem
and I haven't had an issue since.
Has anyone else noticed web bots making web requests? How do others deal
with it? I'm curious.
-spc
[1] https://en.wikipedia.org/wiki/I'm_a_Little_Teapot
Reply to: