[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Stopping webcrawlers.



I doubt that you can get their attention and even if you did I doubt
that they would stop: they may very well be deliberately targeting sites
with a robots.txt.

One reason they may be doing this is in search of vulnerable sites that
they can hijack.

You may need to implement access control.
-- 
John Hasler 
jhasler@newsguy.com
Elmwood, WI USA


Reply to: