russell@coker.com.au (Russell Coker) wrote in message news:<20011223201754.C49A017CB2@lyta.coker.com.au>... > I wasn't aware that there was any format to robots.txt, I thought that the > mere presense of such a file would prevent robots from visiting. Nope; see: http://www.robotstxt.org/wc/robots.html