Re: new, not nice web bots disposal

[Greg Wooledge <wooledg@eeg.ccf.org>]

On Wed, Feb 26, 2020 at 03:57:51AM -0500, Gene Heskett wrote:
> over the last 90 days or so, we seem to have been plauged with a new 
> breed of bots scanning our web pages, and they are not just indexing our 
> web pages I don't mind that, but they are ignoring our robots.txt

I can believe this.  But it's better if you SHOW us the actual logfile
entries (not all of them, obviously -- just a reasonable sample).

If the bots have a recognizable user-agent, you can just configure
your web server to block queries from that user-agent.  Then you don't
have to deal with iptables, nftables, or whatever they're calling
it this year.