[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: rogue Chinese crawler



Once upon a time (actually it was more like Fri, 23 Nov 2001 17:32:04 +0000
(UTC)),
Martin WHEELER <mwheeler@startext.co.uk> wrote:

> Is anyone else having problems with the robot from
> 
>      openfind.com.tw
> 
> -- an intrusive, irritating, hard-to-get-rid-of crawler that completely
> paralyses my system *every day*?
> 
> Despite what I put in any robots.txt, this one disregards all rules and
> just jams up my system, downloading every damn' thing in sight.
> Mails to the owners are totally disregarded.

Have you actually seen:
http://www.openfind.com.tw/robot.html

It talks about the robot and how to get it to stop accessing your site.

Hereward

Attachment: pgpZe7RmysfbV.pgp
Description: PGP signature


Reply to: