Re: On the quest for automated QA checks
Hi, Russ.
I'd like to make it clear that I'm just brainstorming here and that at
this stage, I'm not really concerned with the feasibility of the ideas.
I'm just doing a "what if" thing.
On May 02 2009, Russ Allbery wrote:
> Rogério Brito <rbrito@ime.usp.br> writes:
>
> > * for instance deprecated debhelper programs should generate a big fat
> > warning and, while they don't do that, detecting them.
>
> Lintian should catch this already, although we may not have the most
> recent ones. If you know which are missing, please file a wishlist
> lintian bug.
dh_movefiles doesn't seem to be catched, from tests of today. I don't
know if lintian is really the proper place to put those tests, but it
wouldn't hurt, even though I think that debhelper should generate such
things.
> > * seeing if the rules file has the maintainer overriding, say, CFLAGS
> > and LDFLAGS.
>
> This is hard to analyze properly. The package should not be relying on
> environment variables set by dpkg-buildpackage, since then debian/rules
> build doesn't work.
Hummm, I'm not sure that I get it, but it seems to me that
dpkg-buildpackage already does that, doesn't it? It seems to pass flags
like -O2 -Wall, if I'm not mistaken... Please *do* correct me if I am
wrong.
> > * seeing if a given package providing a library has the -dev package
> > containing the soname in the name of the package (e.g., libfoo0-dev).
>
> This is sometimes the right thing to do, sometimes not.
I know, I know. Just putting a very low priority thing to lintian could
be done...
> > * seeing if the package would benefit from LDFLAGS like --as-needed or
> > -z,defs in the linking stages.
>
> IMO, --as-needed is never a good idea.
>
> It's fairly hard to check except by analyzing a build log whether the
> package is using -z,defs, and it can be very hard to add that sort of
> flag to the upstream build system.
Let me restate here that I'm just brainstorming. Perhaps lintian could
have a category of tests labelled "expensive" tests, where the person
running lintian would be willing to run also a compilation thing in a
chroot/virtual machine/whatever environment.
Perhaps also a "very expensive tests" category could also be included.
> > * seeing if the package properly doesn't have unnecessary dependencies.
>
> Really, really hard to check in an automated way.
Yes, I do know. All these things that I'm proposing here smell like the
"Halting problem", which, as we know, is undecidable. I'm not sure if
those things that I propose here are actually reducible to the "Halting
Problem", but I'm dreaming... :-)
> > * seeing if a -dev package has libc6-dev as one of its dependencies (it
> > seems that the current lintian from unstable doesn't catch this).
>
> It should; please file a bug against lintian.
OK. I will doublecheck it and file the bug, if it is indeed here.
> > * seeing if the long description passes a spell checker test (running
> > ispell or whatever).
>
> Lots and lots of false positives for this. I don't think it's worth it.
This could be in the "expensive" category of tests, I think.
> > * seeing if the files in the debian directory have trailing whitespaces.
>
> Eh, really? I've never understood why people care about this.
Well, I am particularly annoyed by this, but I don't see it as
fundamentally wrong... It is just that if I were a sponsor, I would like
that thing.
> That being said, there's already a wishlist bug against Lintian to
> check for this at the --pedantic level.
Nice. :-)
Oh, perhaps, doing a licensecheck call could be a good thing, if only to
let the maintainer know what is happening and that Debian *does* care
about this.
What I mean with all this is just to get ideas to automate the checks of
packages (making computers do the mechanical work and leaving humans the
finer tasks).
This is all based on the idea that a package that is "syntatically
correct" doesn't necessarily means that it is "correct" (in some very
vague sense of the word).
Thanks for your comments, Rogério.
--
Rogério Brito : rbrito@{mackenzie,ime.usp}.br : GPG key 1024D/7C2CAEB8
http://www.ime.usp.br/~rbrito : http://meusite.mackenzie.com.br/rbrito
Projects: algorithms.berlios.de : lame.sf.net : vrms.alioth.debian.org
Reply to: