Re: swamp rat bots Q
On Thursday 03 December 2020 08:07:33 john doe wrote:
> On 12/3/2020 1:35 PM, Gene Heskett wrote:
> > I've had it with a certain bot that that ignore my robots.txt and
> > proceeds to mirror my site, several times a day, burning up my
> > upload bandwidth. They've moved it to 5 different addresses since
> > midnight.
> >
> > I want to nail the door shut on the first attempted access by these
> > AH's.
> >
> > Does anyone have a ready made script that can watch my httpd "other"
> > log, and if a certain name is at the end of the line, grabs the ipv4
> > src address as arg3 of the line, and applies it to iptables DROP
> > rules?
> >
> > Or do I have to invent a new wheel for this?
> >
> > Basic rules that simplify it somewhat.
> >
> > 1. this is ipv4 only country and not likely to change in the future
> > decade.
> >
> > 2. the list of offending bot names will probably never go beyond 50,
> > if that many. 5 would be realistic.
> >
> > 3. the src address in the log is at a fixed offset, obtainable with
> > the bash MID$ but the dns return will need some acrobatics involving
> > the bash RIGHT$ function.
> >
> > 4. it should track the number of hits, and after so many in a /24
> > block, autoswitch to a /16 block in order to keep the rules file
> > from exploding.
>
> Is that not the same question you asked a while back, I then suggested
> 'fail2ban' or using ip/nftables own capabilities?
>
Yes John. But explain to me what fail2ban is sopposed to do?
Its running, but has failed to ban anything no matter what sort of 403's
I return.
Fail2ban has been running here for years, and in just sits there doing
nothing, so if its as great a swiss army knife as others claim it to be,
lets either make it work, or quit recommending it.
I need something I can feed with a tee off the tail output, detect that
it is one of the offending bots by name, and if so, apply its ipv4
address to an iptables DROP rule.
> It looks to me like you are making your life way much harder than it
> should be.
Fine, now show me how to make fail2ban do something usefull. I just
rebooted because of a drive failure and found it couldn't be found
running by htop, so I started it. Now make it do something usefull if
its so great.
iptables does work but I have to manually pick the addresses out of the
log in order to put them into the rules.
They move these bots around 2 or more times a month to get around people
like me who do use something like iptables, so Ideally I should nuke the
rules file about monthly and restart a new compilation by feeding this
script with the last 5000 lines of the "other" log. And leave the tail
active to feed new hits into this script one line at a time as they
occur.
Thank you.
> --
> John Doe
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>
Reply to: