Re: Test suites
Neil Williams <codehelp@debian.org> writes:
> On Mon, 2008-04-21 at 10:14 -0700, Kevin B. McCarty wrote:
> > Neil Williams wrote:
> > >> 1) Always run the test suite (for example to catch bugs that may not
> > >> occur on the developper's architecture)
> > >
> > > Yes. (That is the main point of having a test suite.)
> >
> > I agree with this (of course with the caveat that
> > DEB_BUILD_OPTIONS=nocheck should disable the test suite). But I'll add
> > a minor and maybe obvious exception -- one should disable or rework any
> > parts of the test suite that would need any of the following:
> >
> > * An X server (e.g. for GUI tests)
> > * A working network connection
> > * Write access to the file system outside the build directory (with
> > obvious exceptions like /tmp)
> > * Interactivity
>
> IMHO test suites that require any of these external requirements are
> simply not sane.
Agreed, if we're talking specifically about a *unit test* suite, as
opposed to the other kinds of tests that should also be in a test
suite: functional tests, stress tests, etc.
Unit tests should go to whatever lengths are necessary to test *only*
the tiny portion of the code that is the subject of the specific test
case, and use test doubles for any external dependencies as listed
above.
With that description of unit tests, it then makes sense to say that
the build daemon should always run the unit test suite. Therefore it
should also be easy to *only* run the unit tests, without any other
tests.
--
\ "Instead of a trap door, what about a trap window? The guy |
`\ looks out it, and if he leans too far, he falls out. Wait. I |
_o__) guess that's like a regular window." -- Jack Handey |
Ben Finney
Reply to: