Fw: Re: Problems when indexing the lists.debian.org site

[Cesar Mendoza <mendoza@debian.org>]

Hi,

So, shall we lift the embargo to Googlebot on lists.debian.org?

Bye
Cesar Mendoza
http://www.kitiara.org
--
"The three golden rules to ensure computer security:
Do not own a computer, do not power it on, and do not use it."
  --Robert T. Morris
----- Forwarded message from Larry Schwimmer <schwim@google.com> -----

> Date: Sun, 7 Jan 2001 15:19:31 -0800 (PST)
> From: Larry Schwimmer <schwim@google.com>
> To: mendoza@debian.org (Cesar Mendoza)
> Cc: googlebot@google.com
> Subject: Re: Problems when indexing the lists.debian.org site
> X-Mailer: ELM [version 2.5 PL3]
> 
> You (Cesar Mendoza) write:
> > Googlebot has been denied access to lists.debian.org, since you guys are
> > not responding to emails about this problem.
> 
> Cesar,
> 
> I never recall seeing your message.  We do respond to requests
> rapidly; I'm sorry yours was missed.
> 
> I decreased the crawl rate for lists.debian.org for future crawls to
> one fifth of the present rate.  We can decrease it further if needed.
> 
> We have about 650k URLs from debian.org and 20% (130k) of those URLs
> are from lists.debian.org.
> 
> -- 
> Larry Schwimmer
> googlebot@google.com
> Google Inc.
> 
> > On Wed, Nov 08, 2000 at 12:20:54PM -0600, Cesar Mendoza wrote:
> > > Is there a way to make googlebot to be more nice when indexing the site
> > > lists.debian.org? lists.debian.org goes down every time you guys hit it.
> 

----- End forwarded message -----