[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Heads up: The Yarn.social (yarns) search engine is crawling twtxt feeds over gopher://



Hi All,

Just a heads up that if you have a twtxt feed you host over gopher:// the Yarn.social search engine (called yarns) hosted at https://search.twtxt.net/ may possibly crawl your feed.

@Cameron Kaiser I apologize in advance, but _some_ users have started following _some_ feeds over your floodgap proxy. To address this I've recently added native support for Gopher feeds to Yarn.social pods (the `yarnd` software) -- If you can help me figure out who/what is still hitting your proxy we can mitigate this and get them migrated off.

The search engine currently re-crawls the entire space, once-per-day. No more at this time.

If you have any concerns or questions don't hesitate to contact me!

Kind regards

James

Reply to: