new, not nice web bots disposal
over the last 90 days or so, we seem to have been plauged with a new
breed of bots scanning our web pages, and they are not just indexing our
web pages I don't mind that, but they are ignoring our robots.txt and
are mirroring anything apache2 can reach, including stuff thats there
but not reachable by a normal browser just looking around and clicking
on links. Its annoying as hell and when you're out in the pucker-brush
on a 10 megabit ADSL, eats up ones available upload bandwidth of about
275kbytes/s. According to my cable billing, these A-H's used over 100Gb
of my bandwidth in Nov 2019. That describes in printable language as a
DDOS in my vocabulary.
So I asked a few questions and wrote some little 2-3 line scripts after
putting a tail on /var/lib/httpd/other_vhosts_access.log, which logs
enough info you can generally identify the bots with it.
I have since have generated 49 iptables rules that have blocked 99% of
them.
Those scripts I've placed in /etc/iptables and are owned by root.
To start iptables after a reboot:You might run this one first one
from /etc/rc.local
root@coyote:iptables$ cat start-iptables
#!/bin/bash
cd /etc/iptables
iptables-restore <rules.v4
To add a new rule, covering that whole 256 address block because they
seem to have a random address, changed about weekly, in that block:
root@coyote:iptables$ cat iptables-add
#!/bin/bash
iptables -I INPUT -s add.ress.to.block/24 -j DROP
Substituting the address of the offender for add.ress.to.block in the
last tine above.
to save the rules:
root@coyote:iptables$ cat iptables-saveem
#!/bin/bash
iptables-save >rules.v4
To see what you've got so far:
root@coyote:iptables$ cat iptables-status
#!/bin/bash
iptables -L -nv --line-numbers
Which will output the rules in effect plus the hits accumulated in this
uptime so far, in this format:
lnum hits bytes fate
24 846 50760 DROP all -- * * 66.249.64.0/24 0.0.0.0/0
Be my guest folks, reclaim the net, we are paying for the bandwidth these
jerks are burning up.
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>
Reply to: