[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: rogue Chinese crawler

Once upon a time (actually it was more like Fri, 23 Nov 2001 17:32:04 +0000
Martin WHEELER <mwheeler@startext.co.uk> wrote:

> Is anyone else having problems with the robot from
>      openfind.com.tw
> -- an intrusive, irritating, hard-to-get-rid-of crawler that completely
> paralyses my system *every day*?
> Despite what I put in any robots.txt, this one disregards all rules and
> just jams up my system, downloading every damn' thing in sight.
> Mails to the owners are totally disregarded.

Have you actually seen:

It talks about the robot and how to get it to stop accessing your site.


Attachment: pgpjATmwodf0Z.pgp
Description: PGP signature

Reply to: