[gopher] Re: Gopher "robots.txt"

[emanuel at heatdeath organisation <emanuel@heatdeath.org>]

On Sun, Jan 14, 2001 at 05:28:31PM -0500, David Allen wrote:
> That's not a bad idea...I'd prefer personally to put whatever
> information that this null statement has in the gopher+ fields to
> avoid overloading the meaning of the other fields (type, name, host,
> port).  So maybe something like this to disallow the debian
> subdirectory:
> 
> iFFerror.hostF909FDisallow: selector_for_debian_directory

Why not put this information in the Gopher+ attribute information?  Have
an attribute that means "ignore this selector" to the robot.  This way
clients that don't know about it will never see it.  If it's set for a
directory, the robot will ignore than directory and will never see
anything liked by the directory.

Isn't this kind of thing what the Gopher+ attribute information is for?

-- 
emanuel at heatdeath organisation
gopher.heatdeath.org