Steve Lamb wrote:
Marty wrote:I don't see what legitimate purpose it might serve, and I wonder if the posters' wishes or search engine users' interests, or even public interests, enter into consideration? I suppose not.The public has no interest.
But google shareholders do? I hope that's not what you mean. The public just happen to be the authors of all the messages on such lists. I don't know why you think they have no interest in their own archives. As for a legitimate reason having constant hits
from search engines could be a problem. Certainly was for my site.
Different issue. I am describing what google seems to be doing with all lists archives and list archive availability, not how or whether it complies with robots.txt files.