On Fri, Aug 08, 2025 at 10:51:21AM +0800, Tianyu Chen wrote:
Yesterday I was packaging python-pywebpush[1]. To enable autopkgtest,
I used `Testsuite: autopkgtest-pkg-pybuild' in d/control. But when
autopkgtest stage, pytest didn't discover any tests. weepingclown and
I dug into the problem and found tests in pywebpush/tests are not
copied to the build directory; instead, pybuild only copies
pyproject.toml, pytest.ini, test, and tests directories. pybuild(1)
documented that "you can list them in debian/pybuild.testfiles file".
By creating d/pybuild.testfiles and adding "pywebpush/tests" to the
file, autopkgtest works. But reviewing the build log, we found all
tests are running twice in dh_auto_test. The log shows:
tests/test_webpush.py
pywebpush/tests/test_webpush.py
I asked on #debian-python IRC yesterday, and currently using `export
PYBUILD_TEST_ARGS=--ignore pywebpush/tests' to avoid being tested
twice. I also did some searching on codesearch.debian.net, and I found
that some other packages are either tested twice or doing similar
things to avoid being tested twice. Following are three attempts to
ensure tested only once:
src:isodate
export PYBUILD_BEFORE_TEST=cp -r {dir}/tests {build_dir}/
export PYBUILD_TEST_ARGS=tests/test_*.py
src:python-cymem
export PYBUILD_NAME = cymem
export PYBUILD_TEST_ARGS = tests
src:python-ewokscore
ifeq ($(PYBUILD_AUTOPKGTEST),1)
export PYBUILD_TEST_ARGS := {dir}/src/ewokscore
endif
Are there better solutions to this problem?
These are all ultimately workarounds for https://bugs.debian.org/947800. I've generally found that there's unfortunately no one size that fits all: which option is best depends on the exact details of the package and how tolerant it is of its tests living in a slightly different location from the one it expects. For example, some tests do relative imports so they'll fail if you effectively move foo/tests/ to tests/, while some tests have expectations about their current working directory relative to the test suite.
With pytest it's often (not always) easiest to tell it to run tests from a non-installed path, as in python-cymem and python-ewokscore above. Manually copying the tests into the expected subdirectory using PYBUILD_BEFORE_TEST and PYBUILD_AFTER_TEST works too, and that's usually what I end up doing with other test frameworks, but it's cumbersome enough that I usually only do this when I have to.
In your case I would probably drop debian/pybuild.testfiles and do `export PYBUILD_TEST_ARGS = {dir}/pywebpush/tests` in debian/rules, although it does produce mildly uglier test output, and to some extent this is a matter of taste.
Ideally I think we'd be able to write something like `pywebpush/tests => pywebpush/tests` in debian/pybuild.testfiles, in the style of dh-exec. Then most of these hacks would be unnecessary.
-- Colin Watson (he/him) [cjwatson@debian.org]