On Fri, Nov 01, 2019 at 01:12:46PM -0400, Gene Heskett wrote: > On Friday 01 November 2019 12:42:21 tomas@tuxteam.de wrote: [...] > > https://en.wikipedia.org/wiki/Robots.txt > > > Did that, blanket Disallow for all didn't stop them. But they are spacing > out the requests now, so the average traffic is very low. I can tolerate > that. You can block by user agent, that's more drastic. If I've been paying attention, you are on apache. Then this might be relevant: https://httpd.apache.org/docs/2.4/rewrite/access.html#blocking-of-robots Cheers -- t
Attachment:
signature.asc
Description: Digital signature