Re: Thoughts on the new Lintian test suite
Russ Allbery wrote:
> Raphael Geissert writes:
>> Russ Allbery wrote:
>
>> What about grouping together tests belonging to the same check?
>> E.g. move all the cruft tests to t/tests/cruft/
>
> We could do that -- it trades off directory size with more nesting. I
> guess I don't have a strong opinion, although either way I'd like to move
> the desc files into the test directory.
Sure
>
>>> * I don't think we need to maintain both test harnesses until we've had a
>>> chance to break the legacy test cases apart into separate,
>>> better-documented test cases. I propose moving all the testset test
>>> cases into the new test suite as legacy-* test cases with a 6600
>>> sequence number so that they run last.
>
>> Why do we need such large numbers anyway? They just look ugly, to me.
>
> Eh, we probably don't, but if it were just a sequence number in the desc
> file, I don't think you'd notice one way or the other.
I'd prefer inside the desc files, less clutter.
>
>> I agree that it is PITA to write new tests and that's why I've been
>> still adding tests to my checks in the testset suite.
>
> Now that I've done a bunch of them, I find it easier to write a new-style
> test case than to add something to testset.
All I used to do was make the change, run that specific test, verify diff, apply
it, re-test, ta-da. The only thing that I didn't like was the specific/strict
sorting of the tags.file.
> Although a helper program
> that prompts you for some of the information to build a new test case and
> its desc file might save a bit more time.
Probably.
>
>> I like the idea of one test per tag, as it allows for isolated and
>> cleaner testing of every check; but we should also have combined tests
>> (i.e. one test for multiple checks to make sure they interoperate fine).
>
> I'd really rather not have a separate test case for every tag, since it
> will make running the full test suite (which I want to do before a
> release) take literally hours. It already takes longer than one would
> want to wait for and is something that one has to do in the background
> while doing other things.
What about parallelising? Another option is to implement a direct analysing
process, i.e. analysing the pseudo packages generated by the test suite without
requiring the .deb files. This would also allow lintian to be run on the source
package without even having to build it, thereby partially implementing
the "lintian should be able to run before building" feature.
It would be a great feature and IIRC it only requires some work on the frontend
and the collect scripts.
Cheers,
--
Raphael Geissert - Debian Maintainer
www.debian.org - get.debian.net
Reply to: