Hi Sebastian, On 28-09-2019 14:04, Sebastian Andrzej Siewior wrote: > the openssl debci testsuite has now a subset (a real small one) compared > to the complete testsuite. I've been looking into running just the > testsuite without recompiling the whole thing but it is not that simple. > There is no cut at the moment. It assumes to use the openssl binary from > the build directory, it assumes a few compiled test programs and > "engines" which usually are in the "engine" folder. One test checks > available symbols which does not work once binary is stripped so I would > have to skip that one. How difficult is it to compile everything but e.g. replace the build binaries with a (soft)link to the installed binaries (and skip tests that aren't appropriate in that case)? I'm not sure if this covers what you meant above. > My question is whether it is okay build the library and everything > required to run the testsuite or not. I've seen some packages doing > this, like the python modules but then compile process takes a fraction > of time compared to the test-suite. This question has multiple sides. Depending on how long the build actually takes versus how much is tested, it's perfectly fine from ci.d.n side to build your package and test it from there (via the build-needed restriction). On the other hand, autopkgtests are really intended to test *as installed* packages. As a third side, because of the way the results influence migration of packages from unstable to testing, also the opinion of the release team matters. Please be reminded that if you autopkgtest fails while testing a reverse dependency, than that reverse dependency will be blocked from migrating. As an example where this went wrong, a lot of packages in the KDE/Qt stack stopped running their autopkgtest because their tests also required building and due to the lock step nature of a lot of their packages, this often failed in the migration setup. Hence it was not robust as autopkgtest and they decided to drop their tests in a number of cases, the latest one I know of is src:kate. That said, if the only reasonable way to substantially test you code is by building the package *and* if this has quite some added value over only doing this during build time, then I don't see a problem to do this in an autopkgtest. Are you able to elaborate on what you hope to achieve? I would encourage you though to discuss this with your upstream such that in some future, you could actually test the installed packages with the test suite, as that is very much desirable. I am not guaranteeing that in the further future we will still allow this. Paul
Attachment:
signature.asc
Description: OpenPGP digital signature