Re: RFC: A regression test framework for Debian packages?
On Mon, Sep 02, 2030 at 11:20:05AM +0200, J?r?me Marant wrote:
> He mentioned that we lack regression tests for packages,
> but as far as I know, no solution has been found.
Yay! 'bout time someone took this particular ball and ran with it!
> What?
> -----
> The main idea is to make tests on package installation
> and after their installation is completed.
Build time is important too: that's the only time you can attempt fine
grained testing of modules rather than executables, and it lets you catch
problems before you upload too. eg, after some irritating bugs that happened
to only show up on one or two architectures, I added some regression tests
to ifupdown, which basically:
(1) made up a fake interfaces file
(2) made up a file with the expected output
(3) invoked ./ifup -nv --force -a -i <blah>
(4) checked the results matched
It's a fairly straightforward shell script, albeit a bit messy and
confusing. At least I can be fairly confident that the particular bug
that caused me enough grief to write all that won't appear again quite
so subtly. It probably only work since ifup can act, essentially, as
a "filter" so you can run it without having to worry about hardware
requirements and such. A lot of programs, and a lot of problems,
that you'd like to have regression tests for can't do that. Dunno if
it matters.
> - tests on maintainer scripts:
> - debconf tests:
> - configuration files tests:
> - application-specific tests:
These all seem pretty hard to actually do. Useful, but hard. Something
that might also be useful, and isn't that hard could be letting you
check that particular files actually end up in the .deb you're creating,
with particular ownerships and permissions.
Cheers,
aj
--
Anthony Towns <aj@humbug.org.au> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. GPG signed mail preferred.
``If you don't do it now, you'll be one year older when you do.''
Reply to: