Hello all,
Thanks to Matthias for pointing this out to me. I try to reply
in-thread. Please keep CC:ing me as I'm not subscribed.
> The goal is to perform a simple “smoke test” to verify the correct
> installation of the Python libraries from the Debian packages, for all
> relevant Python interpreters.
>
> A secondary goal is to have the actual smoke test code be re-usable,
> possibly incorporated into a standard Debian Python packager toolkit.
Very nice! There's precedent for Perl, Ruby and DKMS packages which
have a fairly standard way to run the upstream test suite. For Python
there are some conventions, like "./setup.py test" or running
nosetests, maybe it's worth experimenting with those after getting the
initial "import works" smoketest in place.
This situation (testing many related packages in a similar way) is
what autodep8 is for. When run in a Debian source package tree, it
checks if it knows something about this type of package, and if so,
synthesizes a debian/tests/control. Have a look at
http://anonscm.debian.org/cgit/collab-maint/autodep8.git/tree/
which has a README, and the "detect" and "generate" scripts in
support/.
> * debian/tests/control
>
> Specifies two tests to run, ‘smoke-python2’ and ‘smoke-python3’.
>
> The dependencies explicitly specify the Python 2 and Python 3 library
> modules respectively. Can this be automated better?
The "generate" script could iterate over all binary packages of the
source and pick out the python-* or the python3-* ones, respectively.
> * debian/tests/smoke-python2, debian/tests/smoke-python3
>
> These are Bash programs that just run a Python module under every
> available Python interpreter.
>
> Running ‘py{,3}versions -i’ gets all the installed Python
> interpreters. Can this fail, e.g. can the set of interpreters be
> different from what interpreters the package was built for?
Yes, it can, and that's IMHO a situation where you *want* the test to
fail, as your package is actually broken (or at least in need of a
binNMU).
> These scripts are very simple, but not simple enough that I was
> comfortable stuffing them directly into the ‘debian/tests/control’
> file. Is that a shortcoming of the Autopkgtest specification?
Not sure what you mean with "shortcoming", you can put arbitrary shell
code into Test-Command:. However, the spirit is that these should be
very short, usually one line. I suggest that they should mostly just
call something like "/usr/share/python-something/run-autopkgtest $py $pkgname"
and you maintain the actual test code centrally in e. g.
python-defaults.
> * debian/tests/smoke_test.py
>
> A module that runs a simple suite of smoke tests: emit the interpreter
> description, fetch the Python distribution of the installed library
> and emit that, import the modules and emit those.
>
> Implemented as a command-line program so that the distribution name,
> and the set of module names, are specified by the caller. This
> hopefully makes ‘smoke_test.py’ reuseable for many Python library
> packages.
>
> Does this module meet your idea of a simple “smoke test” for Python
> library packages? Should it do more?
That seems like an excellent starting point indeed. Making sure that
all modules are importable will catch missing builds for particular
Python versions, regressions in newer Python versions, missing
dependencies, packaging errors, etc. As said above, maybe in the
future we want to try and run upstream test suites, but this will need
a lot more heuristics (starting with additional test deps).
Thanks,
Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
Attachment:
signature.asc
Description: PGP signature