Re: Got a puzzle here

[Gene Heskett <gheskett@shentel.net>]

On Friday 01 November 2019 12:42:21 tomas@tuxteam.de wrote:

> On Fri, Nov 01, 2019 at 04:29:38PM -0000, Dan Purgert wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA256
> >
> > Gene Heskett wrote:
> > > [...]
> > > Humm. in /var/log/httpd/other_vhosts_access.log, is looks as if
> > > both www.yandex.com/bots and www.semrush.com/bot.html are trying
> > > to download all of my nitros9 stuff, which is a unix like OS for
> > > the trs-80 color computer.
> >
> > Sounds like you might want a robots.txt file in there, to keep them
> > from grabbing it.
>
> https://en.wikipedia.org/wiki/Robots.txt
>
Did that, blanket Disallow for all didn't stop them. But they are spacing 
out the requests now, so the average traffic is very low. I can tolerate 
that.

Thanks Tomas

> Cheers
> -- t


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>