[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: new, not nice web bots disposal

On Wednesday 26 February 2020 14:57:18 deloptes wrote:

> Gene Heskett wrote:
> > over the last 90 days or so, we seem to have been plauged with a new
> > breed of bots scanning our web pages, and they are not just indexing
> > our web pages I don't mind that, but they are ignoring our
> > robots.txt and are  mirroring anything apache2 can reach, including
> > stuff thats there but not reachable by a normal browser just looking
> > around and clicking on links.  Its annoying as hell and when you're
> > out in the pucker-brush on a 10 megabit ADSL, eats up ones available
> > upload bandwidth of about 275kbytes/s.  According to my cable
> > billing, these A-H's used over 100Gb of my bandwidth in Nov 2019.
> > That describes in printable language as a DDOS in my vocabulary.
> I have same observations at home. I have setup the modem (ADSL) with
> WLAN infront of the firewall. I block everything except 3 ports:
> 80 - for the web server
> 8080 - for the openvpn
> 22222 - for the SSH
> I recently replaced a very old firewall script with shorewall and
> started monitoring the activities in the logs. So it shows too many
> drops,  which I think are port scanners.
> On the server listening on the open ports in apache, openvpn and ssh I
> see also the attacks described.
> The webserver shows content scanners or someone trying to exploit
> services. SSH and openvpn show signs of brute force attacks. I was
> wondering if its normal, but now I think it is and I am sure few
> months ago it was not. Gene is right - it started perhaps 3 months
> ago, while before it was from time to time.
> I can not say how much bandwith gets lost. I do not have much to share
> with the world from this PC :) but it is indeed annoying.
> regards

Whereas i've several gigabytes, much of it could be catalogued as blowing 
my own horn.  So its more than just annoying when there are 100+ 
machines out of the bots that do play by the rules that want to mirror 
the whole thing and have a go get it again rule assuming a 50 gigabit 
pipe, and heavens forbit they wouldn't want to serve up stale data!

And its probably 200% coinkydence that it all started when I first 
published a fully preemptable realtime kernel for an r-pi4b, built and 
running an uptodate buster and debs of linuxcnc built on that pi4b, to 
run on that r-pi4b, almost as if they were trying to punish me for doing 
it.  Not at all plausible, but it does seem like a coincidence.  
Murphy's law, I suppose, gotta have someplace to point my finger while 
sharpening it. ;-)

Thanks deloptes.

Cheers, Gene Heskett
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>

Reply to: