Re: Venting about forums.debian.net

[Jeffrey Walton <noloader@gmail.com>]

On Tue, Jan 20, 2026 at 9:26 AM Svetlana Tkachenko
<svetlana@members.fsf.org> wrote:
>
> > but I suppose you need to compromise with LLM
> > robots going wild.
>
> Are they not required to follow do_not_track http headers or robots.txt ? If LLM robots do not obey these instructions, they should be probably reported to their hosting provider.

Many sites do not honor Do Not Track (DNT) and Global Privacy Control
(Sec-GPC).  I have them set in my browser, and I still get most of the
cookie warnings and Sharing Policy (err, Privacy Policy) violations.
There are notable exceptions, though -- some sites will say something
like "we got your Do Not Track signal, and we are limiting data
collection."  But they are few and far between.

I do not know how well crawlers and bots comply with robots.txt or RFC
9309.  An admin with actual experience should probably comment.

Reporting to the hosting provider is usually a dead end.  I used to
try to contact netblock owners before blocking a range of IP
addresses.  Most of the contact information was missing or incorrect.
The remaining just went unanswered.  Of all the times I sent an email
asking the administrative and technical contacts to contain their
customer, I only received one actual response.

Jeff