[gopher] Strange implementations of "Missing file"

[Damien Carol <damien.carol@gmail.com>]

Hello everybody,

I'm facing a big pb with my search engine/crawler.

The crawler sends selectors that don't exists sometimes. (Updated / removed / don't exists / whatever)

I use to handle error like that :
1) parse response
2) Try to parse as Menu
3) If the first item of the menu is '3' error item type then ERROR

But this method don't works. Mainly because some servers reply very odd responses 

Some old servers sends that :

BAD:
==> 0Sorry, but the requested token could not be found<TAB>Err<TAB>localhost<TAB>70
gopher://wss-ds.no-ip.info:70/0/robots.txt

==> 0'/robots.txt' does not exist<TAB><TAB>error.host<TAB>1
gopher://gdead.berkeley.edu:70/0/robots.txt
gopher://net.bio.net:70/0/robots.txt
gopher://newkraitch.cs.berkeley.edu:70/0/robots.txt
gopher://nemesis.cs.berkeley.edu:70/0/robots.txt
gopher://quix.us:70/0/robots.txt

=> <Empty String>
gopher://sdf.org:79/0/robots.txt

=> finger: /robots.txt: no such user
gopher://holviala.com:79/0/robots.txt


GOOD:
  gopher://gopher.r-36.net:70/0/robots.txt
  gopher://jgw.mdns.org:70/0/robots.txt
  gopher://grids.be:70/0/robots.txt
  gopher://schot.a-eskwadraat.nl:70/0/robots.txt
  gopher://www.quux.org:70/0/robots.txt
  gopher://dams.zapto.org:70/0/robots.txt
  gopher://go.nickshanks.com:70/0/robots.txt


It's very strange. I can understand the "Empty String" could be an easy 
implementation. But others servers sends Menu with 0 items as info... 

Does it old implementations ?

-- 
Damien CAROL
gopher://dams.zapto.org/1/
_______________________________________________
Gopher-Project mailing list
Gopher-Project@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/gopher-project