Bug#915616: libksysguard: 'testsuite' autopkgtest does not test the installed version
Source: libksysguard
Version: 4:5.14.3-1
Severity: normal
The 'testsuite' autopkgtest for libksysguard is done with the build-needed
restriction, does not list any binary packages built by libksysguard as
test dependencies, and is basically a wrapper around dh_auto_test.
It appears this means it is testing a newly-compiled copy of libksysguard
built from the source code. This does not match the intention for
autopkgtests, which is that they test the binaries that are contained in
the .deb distributed by Debian:
*However* note that the tests must test the *installed* version of the
package, as opposed to programs or any other file from the built tree.
— https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst
In particular this allows an autopkgtest for an executable to be used to
detect ABI breaks in one of its dependencies, which could not be detected
if the executable was rebuilt against the updated dependency.
Similar concepts include GNU's `make installcheck`[1] and GNOME's
installed-tests initiative[2].
If the upstream test suite is designed to be run in a built tree at
build-time and test the recently-built binaries, it is likely to need
code changes to become suitable for running against installed binaries.
For example, many of the dbus unit tests have been adapted (borrowing
the GNOME installed-tests conventions) so that they default to
reading data files from ${libexecdir}/installed-tests/dbus and
finding dbus-daemon in the PATH, which means they can be installed in
${libexecdir}/installed-tests/dbus themselves and run from there in order
to test the installed dbus binaries. When run as build-time tests, the
path to the data files and the path to the dbus-daemon (among others)
are overridden by environment variables that point into the build tree.
If KDE doesn't already have a system for doing this, there's nothing
particularly GNOME-specific about the GNOME installed-tests conventions,
so those would be as good a design as any other. The reference test
runner, gnome-desktop-testing, is written with GLib, but there would be
nothing to stop KDE from implementing a compatible runner in Qt/C++ if
that was considered to be a problem - the specification for the metadata
that describes tests is rather simple, and is based on a .desktop-style
file format.
smcv
[1] https://www.gnu.org/prep/standards/html_node/Standard-Targets.html#Standard-Targets
https://www.gnu.org/software/automake/manual/html_node/Install-Tests.html#Install-Tests
[2] https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests
Reply to: