Brilliant, Alex.
---
Delfi Ramirez -- At Work
Mobile: 00 34 633 589231
Email: delfin@segonquart.net
My digital signature
twitter: @delfinramirez
IRC: segonquart
Skype: segonquart
http://segonquart.net
On 2019/05/24 05:58 PM, Alex Schröder wrote:
Here's my write-up. Feel free to use in whatever way you want.
Discussion, edits, clarifications, versioning, updates, I welcome it.
Let's focus on succinctly specifying what V-2 expects, though.
gopher://alexschroeder.ch:70/02019-05-24_Robots_and_Gopher
gophers://alexschroeder.ch:7443/02019-05-24_Robots_and_Gopher
https://alexschroeder.ch/wiki/2019-05-24_Robots_and_Gopher
Personally, my firewall will block bots that don't have a crawl-delay of 1s,
and I can't put that into the robots.txt as specified so if we're going to make
amendments, that would be my first addition. :)
Cheers
Alex