[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: new, not nice web bots disposal



On 2/27/20, tomas@tuxteam.de  wrote:
> On Wed, Feb 26, 2020 at 11:25:53PM -0500, Lee wrote:
>
> [...]
>
>> You're advertising your web server in your sig.  The "other side"
>> ALREADY KNOWS you have a web server there.
>
> If that "other side" is reading your emails, that is.
>
> Not a likely scenario if that "other side" is some malware
> running in some whatever-of-things lightbulb or cat feeder.

This thread is NOT about likely scenarios; we're talking about

| over the last 90 days or so, we seem to have been plauged with a new
| breed of bots scanning our web pages, and they are not just indexing our
| web pages I don't mind that, but they are ignoring our robots.txt and
| are  mirroring anything apache2 can reach, including stuff thats there
| but not reachable by a normal browser just looking around and clicking
| on links.  Its annoying as hell and when you're out in the pucker-brush
| on a 10 megabit ADSL, eats up ones available upload bandwidth of about
| 275kbytes/s.  According to my cable billing, these A-H's used over 100Gb
| of my bandwidth in Nov 2019. That describes in printable language as a
| DDOS in my vocabulary.
|
| So I asked a few questions and wrote some little 2-3 line scripts after
| putting a tail on /var/lib/httpd/other_vhosts_access.log, which logs
| enough info you can generally identify the bots with it.
|
| I have since have generated 49 iptables rules that have blocked 99% of
| them.

**in this case** is it better to have DROP or REJECT on the iptable rules?

I'm saying it might be better to reject than drop.  Watch the logs and
if the A-H's ignore RSTs then go back to drop.

Regards,
Lee


Reply to: