[gopher] Re: Gopher "robots.txt"

[emanuel at heatdeath organisation <emanuel@heatdeath.org>]

> I guess it depends on the robot.  It is sort of what the attribute
> information is for, but should it be the case that a robot has to
> fetch the attribute information for every single directory it ever
> fetches?  That could cause a hit on performance/bandwidth since a
> robot can't fetch just the resource and be safe with that.

It wouldn't be that much of a performance hit.  Rather than fetching the
directory with a normal gopher request, then fetching the attributes for
each entry of the directory, the robot could make a request something
like this:

	1/directoryF$+VIEWS+ABSTRACT+ROBOTS

which would fetch only the VIEWS, ABSTRACT, and ROBOTS attributes of
every item in the directory (in addition to the INFO attribute, which
contains the usual Gopher directory entry).  A fully Gopher+ supporting
robot would need to get the VIEWS and ABSTRACT to build a complete index
anyway.

-- 
emanuel at heatdeath organisation
gopher.heatdeath.org