Re: Got a puzzle here

[Gene Heskett <gheskett@shentel.net>]

On Monday 04 November 2019 09:29:48 Greg Wooledge wrote:

> On Mon, Nov 04, 2019 at 09:08:36AM -0500, Gene Heskett wrote:
> > Does apache2 have its own module that would prevent its responding
> > to an ipv4 address presented in a .conf file as "xx.xx.xx.xx/24"
> > format?
>
> Well, looking at your larger issue, you might find it more useful
> to block these bots based on their user-agent strings.
>
> The first thing you want to do is actually find a log entry from one
> of these bots, so you know what you're dealing with.  If you're not
> logging user-agent, then you'll want to turn that on first.
>
> Once you have that information, you can google "apache block user
> agent" or whatever search terms work best for you.
>
> I'm using nginx on my (real) web site, so I don't have the
> Apache-specific knowledge you're looking for.  I do block one type of
> bot based on its user-agent.  It's pretty simple in nginx:
>
> greg@remote:/etc/nginx$ cat sites-enabled/mywiki.wooledge.org
> server {
>     listen 80;
>     listen 443 ssl;
>     server_name mywiki.wooledge.org;
>
>     if ($http_user_agent ~ SemrushBot) {
>         return 403;
>     }
> ...

And that looks like nginx is a lot easier to program than apache2.  The 
above makes sense. Once I'm functioning again after tomorrows heart 
valve work, I'll investigate that, probably on a fresh drive and a 
buster 10.1 install.  Thanks Greg.

Cheers, Gene Heskett