Hi Chris,
On Wed, Sep 02, 2020 at 10:58:03PM -0000, Chris Lamb wrote:
> Thank you for your work on the Debian Janitor. I was just catching up
> on the progress and it seems like this project has really taken off.
>
> How can Lintian assist these efforts? Would some method of filtering
> Janitor-related issues in Lintian itself be a good first step here?
> I first thought of a usertag, but I have no strong feelings regarding
> the implementation.
Thanks! The existing data provided by lintian has actually been
really useful.
The Janitor uses lintian to find out what issues to fix in which
packages. It also reads some of the lintian data files, such as the
standards version release timestamps and the renamed lintian tags. At
the moment, it will leave any issues that have an explicit override in
e.g. debian/source/lintian-overrides alone.
There was a massive backlog of packages to process, but once we get to
the end of that the Janitor will need to figure out which packages
have issues that it can fix. [1] It gets that information from
the lintian tables in UDD which are currently empty, so I'm looking
forward to seeing #960156 fixed - Felix already has a pull request out
to fix that bug I believe.
There are some things that lintian might be able to help with, or that
may also have relevance for lintian. Most of these are problem
descriptions rather than concrete proposals or they need to be fleshed
out more - but hopefully there's enough here to kickstart some
discussions:
overrides don't communicate their intent
========================================
The Janitor currently checks the overrides files in most cases to see
whether there is an override for a lintian tag; if an override exists,
it will not make any changes.
Unfortunately, lintian overrides are added for a variety of reasons,
the main ones probably being:
1) maintainer disagrees with the issue reported by lintian
2) maintainer doesn't care about the issue reported by lintian
3) false positive from lintian
4) the issue is about upgrading to something new, but the package
needs to be backportable
* ...
There is no way to know why a particular override exists (perhaps
other than parsing the human-readable comments), and so the janitor is
currently just conservative and backs off when there is an override
that is not reported as unused by lintian.
In theory, we should at least be able to do something cleverer for cases
(4) and (2). One (labour-intense and very slow) way to do this would
be to add an extra field with intent - but maybe there are other
ways of inferring intent?
line numbers in tag info
========================
There is no consistent format for line numbers in lintian output -
some tags seem to report them between parentheses "Field blah (line XY)",
some say "Field blah at line XY", etc.
It's not always possible for the Janitor to predict waht the actual
line is that a problem appeared on - due to underlying libraries and
the fact that the files it edits aren't necessarily the same as the
files that end up in the package.
It would be great if the line numbers were stored in a separate field
of the lintian output rather than the "info" field. That way, it's
still possible to e.g. find the relevant override lines without
any fuzzy matches to ignore line numbers.
release compatibility
=====================
Some packages try to stay backwards compatible with older versions of
Debian; that means they'll need to keep doing things in ways
that have since been deprecated and that lintian warns about.
In most cases, this just means that running lintian on the package is
noisy or that the overrides file is very long.
lintian-brush has a configuration option for the latest release it
will try to stay compatible with. This currently lives in a
lintian-brush specific configuration file (debian/lintian-brush.conf),
but it seems like this is something that more tools in Debian (like
lintian) care about. Would it make sense to be able to declare the
latest supported version somewhere else?
verification of fixes
=====================
For most of the lintian issues that it fixes, the janitor will be
able to generate the exact lintian output that matches the issues that
were fixed (perhaps with the exception of line numbers - see above).
At the moment the janitor uses sbuild to build all packages, and that
includes running lintian. It would be great if it could verify that
the issues it thought were being fixed were actually fixed. One
way to do this is if sbuild invocations of lintian would
write a JSON file with the results somewhere (rather than just writing
to standard out), so that it's possible to check that the number of
lintian issues has indeed gone done.
(probably more of a sbuild than a lintian bug request)
Cheers,
Jelmer
[1] https://janitor.debian.net/lintian-fixes/stats
Attachment:
signature.asc
Description: PGP signature