It was thus said that the Great Christoph Lohmann once stated:
> Good point. In eomyidae you have two possibilities:
>
> User-Agent: *
> Disallow: *
Okay, but this diverts from the HTTP version of robots.txt (from my
understanding unless it's been updated since I was last dealing with this
stuff).
> and
>
> User-Agent: *
> Disallow:
This actually has a different meaning from the HTTP version---there this
means "all browsers allowed to crawl" (back from when it robots.txt was
first developed).
-spc