[Pkg-octave-devel] [rafael@debian.org: Re: First try on autopkgtest for OF packages]
[I sent the message below earlier today but it was probably blocked due
to the oversized attached file. The said file is available at
https://wiki.debian.org/Teams/DebianOctaveGroup?action=AttachFile&do=view&target=octave-image-autopkgtest-log.txt]
Rafael
----- Forwarded message from Rafael Laboissière <rafael@debian.org> -----
From: Rafael Laboissière <rafael@debian.org>
Subject: Re: [Pkg-octave-devel] First try on autopkgtest for OF packages
Date: Wed, 30 Aug 2017 16:21:59 +0200
To: pkg-octave-devel@lists.alioth.debian.org
Organization: Debian GNU/Linux
Message-ID: <[🔎] 20170830142159.ho4fijqgzq2dxzuk@laboissiere.net>
* Oliver Heimlich <oheim@posteo.de> [2017-08-30 09:09]:
On 29.08.2017 10:45, Rafael Laboissière wrote:
* Sébastien Villemot <sebastien@debian.org> [2017-08-29 10:28]:
I am also wondering if we should not make test failures fatal
at build time. This would probably cause a few FTBFS in the
short term, but in the longer run this would improve the
quality of the packages.
Yes, absolutely. If a unit test fails, then either the
functionality or the test are wrong and must be fixed.
I totally agree on that one. The interval package (where it is of
particular importance that it produces correct results) has an
extensive test suite and the build fails if one of the tests fails.
Ok, with my last commits on the octave-pkg-dev repository, check-pkg has
now the ability to collect the results of the tests and to exit with an
error code when any of the tests fails (after running all of them, of
course).
With this version of octave-pkg-dev installed in the system, the following
command run from the autopktest branch in the octave-image package:
autopkgtest -B . -- null
produces the log attached below.
Notice that at the end of the log file, there is the following text,
showing that the autopkgtest fails:
Summary: 1169 tests, 1134 passed, 21 known failures, 0 skipped
/usr/share/cdbs/1/class/octave-pkg.mk:108: recipe for target
'check-pkg' failed
make: *** [check-pkg] Error 1
autopkgtest [11:06:09]: test unit-tests: -----------------------]
autopkgtest [11:06:09]: test unit-tests: - - - - - - - - - - results
- - - - - - - - - -
unit-tests FAIL non-zero exit status 2
autopkgtest [11:06:09]: @@@@@@@@@@@@@@@@@@@@ summary
unit-tests FAIL non-zero exit status 2
The code in check-pkg is kind of convoluted. Take this a proof of concept.
Comments are welcome.
Rafael
Reply to: