Re: Got a puzzle here

[Greg Wooledge <wooledg@eeg.ccf.org>]

On Mon, Nov 04, 2019 at 09:08:36AM -0500, Gene Heskett wrote:
> Does apache2 have its own module that would prevent its responding to an 
> ipv4 address presented in a .conf file as "xx.xx.xx.xx/24" format?

Well, looking at your larger issue, you might find it more useful
to block these bots based on their user-agent strings.

The first thing you want to do is actually find a log entry from one
of these bots, so you know what you're dealing with.  If you're not
logging user-agent, then you'll want to turn that on first.

Once you have that information, you can google "apache block user agent"
or whatever search terms work best for you.

I'm using nginx on my (real) web site, so I don't have the Apache-specific
knowledge you're looking for.  I do block one type of bot based on
its user-agent.  It's pretty simple in nginx:

greg@remote:/etc/nginx$ cat sites-enabled/mywiki.wooledge.org 
server {
    listen 80;
    listen 443 ssl;
    server_name mywiki.wooledge.org;

    if ($http_user_agent ~ SemrushBot) {
        return 403;
    }
...