[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: On the quest for automated QA checks

On Sun, 03 May 2009 01:01:51 -0700
Russ Allbery <rra@debian.org> wrote:

> >> > * seeing if a given package providing a library has the -dev package
> >> >   containing the soname in the name of the package (e.g., libfoo0-dev).
> >> This is sometimes the right thing to do, sometimes not.
> > I know, I know. Just putting a very low priority thing to lintian
> > could be done...
> This one's hard -- it's part of a set of tags that one really wants to
> present once for the initial packaging in sort of an "are you really
> sure" way but then not show again.  This is the place where I think
> something on mentors may make a ton of sense, or some way of putting a
> mode into Lintian for "check my non-uploaded package."

Russ, what about a 'lintian -C mentors' check, just as I do with
emdebian-tools? We discussed isolated lintian checks before but it
would make sense for this "one-stage-only" or more precisely
"this-upload-target-only" type tests, just as we use this in Emdebian.
Mentors doesn't have to make the specialised mentors tests fatal by
rejecting the upload unless this is deemed desirable.

The way lintian currently works, that would need to be a secondary test
- i.e. run lintian -iI (possibly with --pedantic) first, then run
'lintian -C mentors' for those specific tests. To be fair to
maintainers, the /usr/share/lintian/checks/mentors file would need to
be packaged for Debian, somehow.

What we really need is that ability to run isolated tests with each
check file being fully insulated from the effects of the others (so
that running 'lintian -C foo' does not run any tests from anything
except /usr/share/lintian/checks/foo).

> > Let me restate here that I'm just brainstorming. Perhaps lintian could
> > have a category of tests labelled "expensive" tests, where the person
> > running lintian would be willing to run also a compilation thing in a
> > chroot/virtual machine/whatever environment.

An isolated -C test is the main method for this kind of operation. The
checks in question are ignored by lintian unless that check is
explicitly specified by -C or possibly by specifying some catch-all
option like --pedantic.
> I wonder if the easy way to do this would be to write something (this
> wouldn't be Lintian) that analyzes a build log for interesting issues.

I've tried build-log parsing for various tasks in Emdebian but it is
actually quite difficult. It generally needs to rely on a specific tool
adding unique tags to the build log for specific stages. I don't think
mentors is ready to require every build to use debuild instead of
dpkg-buildpackage or anything along those lines.

> > Oh, perhaps, doing a licensecheck call could be a good thing, if only
> > to let the maintainer know what is happening and that Debian *does*
> > care about this.
> Yes.  I think running licensecheck for mentors uploads would be great.

'lintian -C mentors' could do that.


Neil Williams

Attachment: pgpBCVNfXe2zm.pgp
Description: PGP signature

Reply to: