Re: Bug#994139: lintian: warning about superficial autopkgtests is counterproductive
Hi,
On Sun, Sep 12, 2021 at 3:27 PM Simon McVittie <smcv@debian.org> wrote:
>
> the first of those is the new missing-tests-control, and I would
> agree with making it an error.
Done. It is now an error. [1]
I'll note that the condition is somewhat artificial. It would probably
be better to declare all testing matters—like deployment of a team's
test suite—in d/tests/control. The mere presence of that file would
declare the test suite. The condition could then never occur,
rendering the tag obsolete.
[1] https://salsa.debian.org/lintian/lintian/-/commit/79c47559e95213c4318b7e02130aa962332c75ea
> I think the second of those is the old testsuite-autopkgtest-missing,
> which apparently no longer exists. Saying that testsuite-autopkgtest-missing
> is an old name for the new missing-tests-control seems inaccurate?
The old tag testsuite-autopkgtest-missing tested somewhat
indiscriminately whether 'autopkgtest' was mentioned in the field
Testsuite in d/control. [2] Dpkg since version 1.17.11, however,
automatically adds that value to the dsc when d/tests/control is
present [3]. That part of the condition is no longer meaningful. (It
was arguably also the only error detected by the tag.)
For the other half of the tag—and the purpose stated in its
description—namely "package does not declare a test suite" [3] there
is from what I can tell no project consensus. That is why the tag was
deleted.
As a side note, the old tag to which you seem attached took a stance
against superficial tests. [5]
[2] https://salsa.debian.org/lintian/lintian/-/commit/c81fb3dbc4425c4322c67f0f3eeb2c208e337736#6f4367b101cfbfb8143c3faa1edd9c67326e2614_76_70
[3] https://sources.debian.org/src/dpkg/1.20.9/debian/changelog/
[4] https://salsa.debian.org/lintian/lintian/-/commit/c81fb3dbc4425c4322c67f0f3eeb2c208e337736#339cb554585174385393ac6779bbc6202a1339b4_4_0
[5] https://salsa.debian.org/lintian/lintian/-/commit/c81fb3dbc4425c4322c67f0f3eeb2c208e337736#339cb554585174385393ac6779bbc6202a1339b4_19_0
> Conversely, if superficial-tests is serious enough to justify a lintian
> tag, then testsuite-autopkgtest-missing should be a lintian tag of equal
> or higher severity, because it's a strictly lower level of test coverage
> than superficial-tests.
I agree with you here, but that logic does not hold universally. The
level of coverage is only one of several criteria that apply when
settling on a tag's appropriate visibility. For example, there may be
a stronger project consensus that tests should be meaningful rather
than that tests should be required.
> to avoid
> "noise" for the maintainers of packages where autopkgtest coverage is not
> feasible to implement, then perhaps they should be classification tags?
Classification tags are used for research in the archive. They are
examined via our website's JSON interface and can yield helpful
information about relative prevalence ratios. They cannot reduce noise
for certain package families. For that, you may be excited about a new
feature in Lintian. [6] You can now request screens for your package
family although there are some hurdles to being so recognized. [7]
In response to Paul's email, which came in the meantime, the screens
can be used to mask the tag superficial-tests (or any other
autopkgtest tag discussed here) for sources for which meaningful tests
do not exist.
[6] https://salsa.debian.org/lintian/lintian/-/commit/8c3d587559cfbdf5dd41e9ba1f66c9ab52de577e
[7] https://lintian.debian.org/screens
Kind regards
Felix Lechner
Reply to: