[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Got a puzzle here



On Fri, Nov 01, 2019 at 04:29:38PM -0000, Dan Purgert wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> Gene Heskett wrote:
> > [...]
> > Humm. in /var/log/httpd/other_vhosts_access.log, is looks as if both 
> > www.yandex.com/bots and www.semrush.com/bot.html are trying to download 
> > all of my nitros9 stuff, which is a unix like OS for the trs-80 color 
> > computer.
> 
> Sounds like you might want a robots.txt file in there, to keep them from
> grabbing it.

https://en.wikipedia.org/wiki/Robots.txt

Cheers
-- t

Attachment: signature.asc
Description: Digital signature


Reply to: