Hi,
Adding debian-python@l.d.o
The context is #1000803 where Sandro asked about reducing duplication
when running upstream test suites both during the build and under
autopkgtest.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000803
On Tue, Nov 30, 2021 at 12:47:42AM -0500, Sandro Tosi wrote:
> > This is usually solved outside of autopkgtest itself.
> >
> > For example Ruby packages have a single place where tests are specified
> > (debian/ruby-tests.*{rb,rake,yaml}, and the appropriate
> > debian/tests/control file is autogenerated by autodep8. I would say 99%
> > of Ruby packages require no duplication, or even explicitly specifying
> > commands to run tests at all.
> >
> > In general, this is most efficiently solved by package type (e.g.
> > programming language), because the way the tests are run at build-time
> > usually is tied to the build system. You then usually need some test
> > runner, probably extracted from, or provided by, the build system, to
> > also provide the same funcionality in an autopkgtest-compatible way.
> >
> > All the package types that are well supported in autodep8 have their
> > specialized test runner tools that avoid this type of duplication.
>
> I'm mostly interested in the python ecosystem, and the provided
> autodep8 test are extremely superficial (essentially only importing
> the main python module packaged), which is better than nothing but not
> the same level as what's in d/rules.
>
> Are you aware of any effort to expand the Python autodep8 tests to
> uniform them into a comprehensive, and unique place to run tests at
> build-time and via autopkgtest?
I'm not.
> If no such effort currently exists, what would one have to do to start
> developing it? Not necessarily volunteering, just gathering
> information.
In general terms, I see this being implemented like this:
0) modify pybuild so one can call e.g. `pybuild --autopkgtest`. That
should work almost the same as `pybuild --test`, but would copy the test
files under ${AUTOPKGTEST_TMP} and run the tests from there, instead of
assuming a previously built tree and trying to chdir into
.pybuild/*/build.
1) add a way of being able to specify pybuild parameters outside of
debian/rules that can be used both during the build and under
autopkgtest
For Ruby, gem2deb-test-runner is driven by debian/ruby-tests.*, both
during the build, and under autopkgtest.
In pybuild, some aspects of the test run can already be done this
way, e.g. debian/pybuild.testfiles. Maybe we could have
debian/pybuild.env to read options in the same way as they are set in
debian/rules today (with environment variables).
2) update autodep8 to generate the appropriate control file to call
`pybuild --autopkgtest`. This needs to be "smart" enough to not
automatically add this call to packages that are not ready for it. At
the moment, almost 1000 source packages have
`Testsuite: autopkgtest-pkg-python`. We would probably need a new
value, or (probably better) to additionally check for something else
in the source tree.
Each item has quite some details to be figured out, but this is the
general idea.
If someone is willing to work on this I could help with guidance along
the way.
Attachment:
signature.asc
Description: PGP signature