[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: swamp rat bots Q

On Thursday 03 December 2020 08:07:33 john doe wrote:

> On 12/3/2020 1:35 PM, Gene Heskett wrote:
> > I've had it with a certain bot that that ignore my robots.txt and
> > proceeds to mirror my site, several times a day, burning up my
> > upload bandwidth. They've moved it to 5 different addresses since
> > midnight.
> >
> > I want to nail the door shut on the first attempted access by these
> > AH's.
> >
> > Does anyone have a ready made script that can watch my httpd "other"
> > log, and if a certain name is at the end of the line, grabs the ipv4
> > src address as arg3 of the line, and applies it to iptables DROP
> > rules?
> >
> > Or do I have to invent a new wheel for this?
> >
> > Basic rules that simplify it somewhat.
> >
> > 1. this is ipv4 only country and not likely to change in the future
> > decade.
> >
> > 2. the list of offending bot names will probably never go beyond 50,
> > if that many. 5 would be realistic.
> >
> > 3. the src address in the log is at a fixed offset, obtainable with
> > the bash MID$ but the dns return will need some acrobatics involving
> > the bash RIGHT$ function.
> >
> > 4. it should track the number of hits, and after so many in a /24
> > block, autoswitch to a /16 block in order to keep the rules file
> > from exploding.
> Is that not the same question you asked a while back, I then suggested
> 'fail2ban' or using ip/nftables own capabilities?
Yes John. But explain to me what fail2ban is sopposed to do?

Its running, but has failed to ban anything no matter what sort of 403's 
I return.

Fail2ban has been running here for years, and in just sits there doing 
nothing, so if its as great a swiss army knife as others claim it to be, 
lets either make it work, or quit recommending it. 

I need something I can feed with a tee off the tail output, detect that 
it is one of the offending bots by name, and if so, apply its ipv4 
address to an iptables DROP rule.

> It looks to me like you are making your life way much harder than it
> should be.

Fine, now show me how to make fail2ban do something usefull. I just 
rebooted because of a drive failure and found it couldn't be found 
running by htop, so I started it. Now make it do something usefull if 
its so great.

iptables does work but I have to manually pick the addresses out of the 
log in order to put them into the rules.

They move these bots around 2 or more times a month to get around people 
like me who do use something like iptables, so Ideally I should nuke the 
rules file about monthly and restart a new compilation by feeding this 
script with the last 5000 lines of the "other" log.  And leave the tail 
active to feed new hits into this script one line at a time as they 

Thank you.
> --
> John Doe

Cheers, Gene Heskett
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

Reply to: