Re: Thoughts on the new Lintian test suite
Hi,
Russ Allbery wrote:
>
> * The t/tests directory is getting quite large. One easy thing we could
> do that would help this, and which I think would also make sense given
> the rest of the layout, is to move the *.desc file for a test from
> t/tests into the test directory (calling it desc or something) and then
> adding Sequence as a required field.
What about grouping together tests belonging to the same check?
E.g. move all the cruft tests to t/tests/cruft/
[...]
>
> * I don't think we need to maintain both test harnesses until we've had a
> chance to break the legacy test cases apart into separate,
> better-documented test cases. I propose moving all the testset test
> cases into the new test suite as legacy-* test cases with a 6600
> sequence number so that they run last.
Why do we need such large numbers anyway? They just look ugly, to me.
>
> I should also note, as I mentioned in the documentation, that I've not
> been sticking to one test per tag, since if we do that we're going to end
> up with a truly amazing number of tests. Instead, when a set of tests
> seem closely related, I've been combining them into one test case,
> sometimes one that generates multiple binary packages if that's needed to
> test a variety of things. Examples are control-file-general, which tests
> a variety of syntax errors and problems in debian/control, or
> copyright-file-general, which generates a bunch of binary packages with
> different copyright problems. I find this structure a bit simpler than
> one test case per tag since it makes it a little easy to add, say, a new
> copyright file by just creating a new file in the debian/ directory and
> adding a new stanza to debian/control.in.
I agree that it is PITA to write new tests and that's why I've been still adding
tests to my checks in the testset suite.
I like the idea of one test per tag, as it allows for isolated and cleaner
testing of every check; but we should also have combined tests (i.e. one test
for multiple checks to make sure they interoperate fine).
Cheers,
--
Raphael Geissert - Debian Maintainer
www.debian.org - get.debian.net
Reply to: