Re: Thoughts on the new Lintian test suite
Raphael Geissert <atomo64+debian@gmail.com> writes:
> Russ Allbery wrote:
>> * The t/tests directory is getting quite large. One easy thing we could
>> do that would help this, and which I think would also make sense given
>> the rest of the layout, is to move the *.desc file for a test from
>> t/tests into the test directory (calling it desc or something) and then
>> adding Sequence as a required field.
> What about grouping together tests belonging to the same check?
> E.g. move all the cruft tests to t/tests/cruft/
We could do that -- it trades off directory size with more nesting. I
guess I don't have a strong opinion, although either way I'd like to move
the desc files into the test directory.
>> * I don't think we need to maintain both test harnesses until we've had a
>> chance to break the legacy test cases apart into separate,
>> better-documented test cases. I propose moving all the testset test
>> cases into the new test suite as legacy-* test cases with a 6600
>> sequence number so that they run last.
> Why do we need such large numbers anyway? They just look ugly, to me.
Eh, we probably don't, but if it were just a sequence number in the desc
file, I don't think you'd notice one way or the other.
> I agree that it is PITA to write new tests and that's why I've been
> still adding tests to my checks in the testset suite.
Now that I've done a bunch of them, I find it easier to write a new-style
test case than to add something to testset. Although a helper program
that prompts you for some of the information to build a new test case and
its desc file might save a bit more time.
> I like the idea of one test per tag, as it allows for isolated and
> cleaner testing of every check; but we should also have combined tests
> (i.e. one test for multiple checks to make sure they interoperate fine).
I'd really rather not have a separate test case for every tag, since it
will make running the full test suite (which I want to do before a
release) take literally hours. It already takes longer than one would
want to wait for and is something that one has to do in the background
while doing other things.
I think we have some groups of tests that are obviously independent that
are easy to bunch together, like most of the files tags.
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
Reply to: