Re: new, not nice web bots disposal
On Thursday 27 February 2020 10:07:18 Lee wrote:
> On 2/27/20, tomas@tuxteam.de wrote:
> > On Wed, Feb 26, 2020 at 11:25:53PM -0500, Lee wrote:
> >
> > [...]
> >
> >> You're advertising your web server in your sig. The "other side"
> >> ALREADY KNOWS you have a web server there.
> >
> > If that "other side" is reading your emails, that is.
> >
> > Not a likely scenario if that "other side" is some malware
> > running in some whatever-of-things lightbulb or cat feeder.
>
> This thread is NOT about likely scenarios; we're talking about
>
> | over the last 90 days or so, we seem to have been plauged with a new
> | breed of bots scanning our web pages, and they are not just indexing
> | our web pages I don't mind that, but they are ignoring our
> | robots.txt and are mirroring anything apache2 can reach, including
> | stuff thats there but not reachable by a normal browser just looking
> | around and clicking on links. Its annoying as hell and when you're
> | out in the pucker-brush on a 10 megabit ADSL, eats up ones available
> | upload bandwidth of about 275kbytes/s. According to my cable
> | billing, these A-H's used over 100Gb of my bandwidth in Nov 2019.
> | That describes in printable language as a DDOS in my vocabulary.
> |
> | So I asked a few questions and wrote some little 2-3 line scripts
> | after putting a tail on /var/lib/httpd/other_vhosts_access.log,
> | which logs enough info you can generally identify the bots with it.
> |
> | I have since have generated 49 iptables rules that have blocked 99%
> | of them.
>
> **in this case** is it better to have DROP or REJECT on the iptable
> rules?
>
> I'm saying it might be better to reject than drop. Watch the logs and
> if the A-H's ignore RSTs then go back to drop.
>
> Regards,
> Lee
Okkaaayyyy. I rebooted about 14 hours back and restarted after switching
it all to reject which says is now sending a "reject-with
icmp-port-unreachable" msg.
If they are obeying the REJECT, one hit should do it, right? In 14 hours
of uptime, 4 have hit a given rule more than once, up to 9 times. That
looks like they are ignoreing the REJECT's to me. The overall traffic
is more frequent, but is not at a nuisance level, yet.
Thanks Lee
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>
Reply to: