[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Auto reject if autopkgtest of reverse dependencies fail or cause FTBFS

On Jan 16, 2017, at 05:45 PM, Russ Allbery wrote:

>autopkgtest is useful for adding additional tests of the built binaries,
>but I don't believe it's intended as a replacement for build-time testing.
>Maybe I've missed something?

No, I think you're exactly right.  If an upstream provides unit tests, those
are totally appropriate to run at build time -and to fail the build if they
fail- but may not be appropriate to run in autopkgtest.  autopkgtests should
be reserved for larger suitability tests on the built and installed package.

An example might be a Python library's test suite.  It makes sense to run
these at build time because that's usually when upstream will run them
(i.e. during development of the package).  But since the test suite usually
isn't run on a built package, it shouldn't be autopkgtested.  The environment
for build tests and autopkgtests are importantly different, e.g. the former
does not/should not allow access to the internet while the latter can and
sometimes must.  A good example of an autopkgtest would be an import test for
a Python module, i.e. once the package is built and installed, does it import?
In fact, autodep8 will automatically add import tests for Python modules in a
safe way (by cd'ing to a temporary directory first).

There are occasionally good reasons why an upstream's test suite can't be run
at build-time, and in those few cases I will run them in an autopkgtest.  But
generally, I think the two are there to test different aspects or lifecycles
of the package.


Attachment: pgpCiUh1SX7NP.pgp
Description: OpenPGP digital signature

Reply to: