[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: new, not nice web bots disposal

On Thursday 27 February 2020 10:07:18 Lee wrote:

> On 2/27/20, tomas@tuxteam.de  wrote:
> > On Wed, Feb 26, 2020 at 11:25:53PM -0500, Lee wrote:
> >
> > [...]
> >
> >> You're advertising your web server in your sig.  The "other side"
> >> ALREADY KNOWS you have a web server there.
> >
> > If that "other side" is reading your emails, that is.
> >
> > Not a likely scenario if that "other side" is some malware
> > running in some whatever-of-things lightbulb or cat feeder.
> This thread is NOT about likely scenarios; we're talking about
> | over the last 90 days or so, we seem to have been plauged with a new
> | breed of bots scanning our web pages, and they are not just indexing
> | our web pages I don't mind that, but they are ignoring our
> | robots.txt and are  mirroring anything apache2 can reach, including
> | stuff thats there but not reachable by a normal browser just looking
> | around and clicking on links.  Its annoying as hell and when you're
> | out in the pucker-brush on a 10 megabit ADSL, eats up ones available
> | upload bandwidth of about 275kbytes/s.  According to my cable
> | billing, these A-H's used over 100Gb of my bandwidth in Nov 2019.
> | That describes in printable language as a DDOS in my vocabulary.
> |
> | So I asked a few questions and wrote some little 2-3 line scripts
> | after putting a tail on /var/lib/httpd/other_vhosts_access.log,
> | which logs enough info you can generally identify the bots with it.
> |
> | I have since have generated 49 iptables rules that have blocked 99%
> | of them.
> **in this case** is it better to have DROP or REJECT on the iptable
> rules?
> I'm saying it might be better to reject than drop.  Watch the logs and
> if the A-H's ignore RSTs then go back to drop.
> Regards,
> Lee

Okkaaayyyy. I rebooted about 14 hours back and restarted after switching 
it all to reject which says is now sending a "reject-with 
icmp-port-unreachable" msg.

If they are obeying the REJECT, one hit should do it, right?  In 14 hours 
of uptime, 4 have hit a given rule more than once, up to 9 times.  That 
looks like they are ignoreing the REJECT's to me.  The overall traffic 
is more frequent, but is not at a nuisance level, yet.

Thanks Lee

Cheers, Gene Heskett
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

Reply to: