Re: swamp rat bots Q
On Thursday 03 December 2020 08:07:33 john doe wrote:
> On 12/3/2020 1:35 PM, Gene Heskett wrote:
> > I've had it with a certain bot that that ignore my robots.txt and
> > proceeds to mirror my site, several times a day, burning up my
> > upload bandwidth. They've moved it to 5 different addresses since
> > midnight.
> > I want to nail the door shut on the first attempted access by these
> > AH's.
> > Does anyone have a ready made script that can watch my httpd "other"
> > log, and if a certain name is at the end of the line, grabs the ipv4
> > src address as arg3 of the line, and applies it to iptables DROP
> > rules?
> > Or do I have to invent a new wheel for this?
> > Basic rules that simplify it somewhat.
> > 1. this is ipv4 only country and not likely to change in the future
> > decade.
> > 2. the list of offending bot names will probably never go beyond 50,
> > if that many. 5 would be realistic.
> > 3. the src address in the log is at a fixed offset, obtainable with
> > the bash MID$ but the dns return will need some acrobatics involving
> > the bash RIGHT$ function.
> > 4. it should track the number of hits, and after so many in a /24
> > block, autoswitch to a /16 block in order to keep the rules file
> > from exploding.
> Is that not the same question you asked a while back, I then suggested
> 'fail2ban' or using ip/nftables own capabilities?
Yes John. But explain to me what fail2ban is sopposed to do?
Its running, but has failed to ban anything no matter what sort of 403's
Fail2ban has been running here for years, and in just sits there doing
nothing, so if its as great a swiss army knife as others claim it to be,
lets either make it work, or quit recommending it.
I need something I can feed with a tee off the tail output, detect that
it is one of the offending bots by name, and if so, apply its ipv4
address to an iptables DROP rule.
> It looks to me like you are making your life way much harder than it
> should be.
Fine, now show me how to make fail2ban do something usefull. I just
rebooted because of a drive failure and found it couldn't be found
running by htop, so I started it. Now make it do something usefull if
its so great.
iptables does work but I have to manually pick the addresses out of the
log in order to put them into the rules.
They move these bots around 2 or more times a month to get around people
like me who do use something like iptables, so Ideally I should nuke the
rules file about monthly and restart a new compilation by feeding this
script with the last 5000 lines of the "other" log. And leave the tail
active to feed new hits into this script one line at a time as they
> John Doe
Cheers, Gene Heskett
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>