[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Compile the package and run the testsuite as part of the debci

Hi Sebastian,

On 28-09-2019 14:04, Sebastian Andrzej Siewior wrote:
> the openssl debci testsuite has now a subset (a real small one) compared
> to the complete testsuite. I've been looking into running just the
> testsuite without recompiling the whole thing but it is not that simple.
> There is no cut at the moment. It assumes to use the openssl binary from
> the build directory, it assumes a few compiled test programs and
> "engines" which usually are in the "engine" folder. One test checks
> available symbols which does not work once binary is stripped so I would
> have to skip that one.

How difficult is it to compile everything but e.g. replace the build
binaries with a (soft)link to the installed binaries (and skip tests
that aren't appropriate in that case)? I'm not sure if this covers what
you meant above.

> My question is whether it is okay build the library and everything
> required to run the testsuite or not. I've seen some packages doing
> this, like the python modules but then compile process takes a fraction
> of time compared to the test-suite.

This question has multiple sides. Depending on how long the build
actually takes versus how much is tested, it's perfectly fine from
ci.d.n side to build your package and test it from there (via the
build-needed restriction). On the other hand, autopkgtests are really
intended to test *as installed* packages.

As a third side, because of the way the results influence migration of
packages from unstable to testing, also the opinion of the release team
matters. Please be reminded that if you autopkgtest fails while testing
a reverse dependency, than that reverse dependency will be blocked from
migrating. As an example where this went wrong, a lot of packages in the
KDE/Qt stack stopped running their autopkgtest because their tests also
required building and due to the lock step nature of a lot of their
packages, this often failed in the migration setup. Hence it was not
robust as autopkgtest and they decided to drop their tests in a number
of cases, the latest one I know of is src:kate.

That said, if the only reasonable way to substantially test you code is
by building the package *and* if this has quite some added value over
only doing this during build time, then I don't see a problem to do this
in an autopkgtest. Are you able to elaborate on what you hope to achieve?

I would encourage you though to discuss this with your upstream such
that in some future, you could actually test the installed packages with
the test suite, as that is very much desirable. I am not guaranteeing
that in the further future we will still allow this.


Attachment: signature.asc
Description: OpenPGP digital signature

Reply to: