Re: Spam: process the web archives?
Le Wed, Oct 16, 2002 at 10:15:07PM -0400, Joey Hess écrivait:
> Adam Lazur wrote:
> > FWIW, Apple serves up their their list archives at lists.apple.com only
> > to users who have http auth'd. The http auth prompt is: "use 'archives',
> > 'archives' or read FAQ for more info".
> >
> > I don't know how well it actually works though.
>
> That's a reasonable trick, that would probably work ok for our user-base
> (need to check how various text browers display such a http auth
> though).
The problem is that google (and other crawlers) would not index our
list archives which are very useful ... and also our only good way of
doing research on our mailing lists at the moment.
Unless you manage to remove the http auth for particular
user-agent (ie the most popular crawlers).
Cheers,
--
Raphaël Hertzog -+- http://www.ouaza.com
Formation Linux et logiciel libre : http://www.logidee.com
Reply to: