[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

new, not nice web bots disposal

over the last 90 days or so, we seem to have been plauged with a new 
breed of bots scanning our web pages, and they are not just indexing our 
web pages I don't mind that, but they are ignoring our robots.txt and 
are  mirroring anything apache2 can reach, including stuff thats there 
but not reachable by a normal browser just looking around and clicking 
on links.  Its annoying as hell and when you're out in the pucker-brush 
on a 10 megabit ADSL, eats up ones available upload bandwidth of about 
275kbytes/s.  According to my cable billing, these A-H's used over 100Gb 
of my bandwidth in Nov 2019. That describes in printable language as a 
DDOS in my vocabulary.

So I asked a few questions and wrote some little 2-3 line scripts after 
putting a tail on /var/lib/httpd/other_vhosts_access.log, which logs 
enough info you can generally identify the bots with it.

I have since have generated 49 iptables rules that have blocked 99% of 

Those scripts I've placed in /etc/iptables and are owned by root.
To start iptables after a reboot:You might run this one first one 
from /etc/rc.local

root@coyote:iptables$ cat start-iptables

cd /etc/iptables
iptables-restore <rules.v4

To add a new rule, covering that whole 256 address block because they 
seem to have a random address, changed about weekly, in that block:

root@coyote:iptables$ cat iptables-add

iptables -I INPUT -s add.ress.to.block/24 -j DROP

Substituting the address of the offender for add.ress.to.block in the 
last tine above.

to save the rules:
root@coyote:iptables$ cat iptables-saveem

iptables-save >rules.v4

To see what you've got so far:
root@coyote:iptables$ cat iptables-status

iptables -L -nv --line-numbers

Which will output the rules in effect plus the hits accumulated in this 
uptime so far, in this format:
lnum   hits  bytes fate
24     846   50760 DROP  all  --  *  *

Be my guest folks, reclaim the net, we are paying for the bandwidth these 
jerks are burning up.

Cheers, Gene Heskett
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

Reply to: