Checklist request (was: RFC: Deb 2.0 testing process)

[Ian Jackson <ian@chiark.greenend.org.uk>]

Philip Hands:
> It seems a shame to have to ask people to do this sort of thing.
> 
> It strikes me that one should be able to come up with a script that
> does a test of this sort in not much more that the time required to
> write the list (in this simple case at least ;-)
> 
> I really think we should encourage people to do this where possible.

I agree.  These tests should be shipped with the package source (not
in the .deb file, since most users won't want them).

Here's a draft scheme of things:

Each source package has an executable file
 debian/tests/test_<binary-package>
for each binary package.  This program, when run with debian/tests as
its current directory (which may contain other files used for the
tests), either:
 produces an exit status of 0 and some possibly-irrelevant data on
   stderr which can be ignored by the caller
 produces an exit status !=0 some stderr output which tells humans
    what went wrong
It may also print to stdout lines of the form
  desire DEBIAN_TEST_*
  require DEBIAN_TEST_*
indicating that a more meaningful test can be done if the relevant
variable(s) below are set, or that no meaningful test can be done if
they are not.

The test script must be invoked with the Source-Depends of the source
package satisfied.

In all cases the test script may create files starting with `tmp_',
which must be cleaned up by its caller.  The test script may not
create other files.  It may never modify the source package of which
it is a part, or other associated files and directories, even if
DEBIAN_TEST_SCRATCH is set.

Its behaviour is modified by at least these environment variables:
  DEBIAN_TEST_SCRATCH
    if set to a nonempty string then the test is allowed to randomly
    mess with the system, including installing or removing packages,
    reconfiguring things randomly, eating mail belonging to the user
    that invokes it, etc.  Otherwise it is not.  Test scripts should
    try to avoid the necessity for this.
  DEBIAN_TEST_GAINROOT
    if set to a nonempty string then the test is allowed to become
    root by invoking DEBIAN_TEST_GAINROOT in front of its arguments.
    DEBIAN_TEST_GAINROOT may not contain spaces or other shell
    metacharacters.  (A la dpkg-buildpackage.)
  DEBIAN_TEST_RECOMMENDED
    indicates that the packages which are Recommended by the binary
    package in question are installed
  DEBIAN_TEST_SUGGESTED
    indicates that the packages which are Suggested by the binary
    package in question are installed.  It is not permitted for a
    package to require this option for meaningful testing using
       require DEBIAN_TEST_SUGGESTED
    because Suggested packages may not be in main.

These scripts could be written by package maintainers, or by the
testing group and submitted as bugs.

Ian.


--
TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word "unsubscribe" to
debian-devel-request@lists.debian.org . 
Trouble?  e-mail to templin@bucknell.edu .