Bug#935706: lintian: Make tag certainty a programmatic assessment
"Chris Lamb" <lamby@debian.org> writes:
> Controversial opinion — the "certainty" of tags is of no actionable
> benefit to either the users of Lintian or its developers and should be
> removed.
I may be able to provide a bit of historical context here, since I was
maintaining Lintian when this was introduced. This is not disagreement
with your proposal (I find your argument persuasive), just context so that
you know what problem we were trying to solve.
Prior to certainty, Lintian set the level of the tags directly. However,
they didn't have any well-defined meaning, and there was some confusion
and controversy over why a given tag would have a given level.
It seemed natural to tie the severity of the tag to the severity of the
bug that would be filed were one to file bugs based on lintian tags, since
that was tied into other project work and judgments (policy, BTS
conventions, and so forth). Debian maintainers already understood the
difference between serious, important, and normal, so reusing that
terminology seemed wise and it would put the level of each tag on more
concrete footing.
The problem, though, was that in some cases the bug would be a serious
Policy violation *if Lintian were right*, but Lintian was often wrong.
Certainty was an attempt to somehow capture that so that Lintian could
express to the maintainer "this is a serious problem with your package if
what I found is true, but there's a good chance this is a false positive."
The "certain" severity was always a problem; only a few things are truly
certain, since there are always special exceptions, and that's always
annoyed people. From my perspective, the certainty concept is more useful
for the wild-guess end of the spectrum, where it's conveying useful
information ("this would be a serious problem, but we're bad at reliably
detecting it").
A theory at the time was that maintainers could use these two metrics to
filter output on. Maintainers that only cared about actual Policy
violations, not Lintian's various advice, could filter on severity of
normal (or important), but allow any certainty. Maintainers who wanted
the advice but didn't want to be bothered with false positives could
enable any severity but filter out certainty of wild-guess and maybe even
possible.
In practice, I don't think this has happened. My impression is that the
classification system is more fine-grained than the users of Lintian care
about, so maintaining it is to some extent wasted effort.
--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>
Reply to: