Re: running CI tests before package sponsorship
Hello,
Thanks for the response. Indeed unit tests are already running during
package build on my local machine and I use autopkgtest to test the
installed packages for proper functioning.
However this only allows me to test on x86_64. My concern was more
that I was under the impression that Debian packages get automatically
tested on various architectures, sometimes a bit esoteric, like Sparc,
etc.
I am already trying to determine which tests in upstream are sensitive
to different versions of BLAS etc and to skip them or make them
non-fatal. There are about a thousand tests that are run and while
they all generally pass, between devs on the project there have been
varying reports depending on operating systems for example, so I am
expecting surprises ;)
(To be clear, I would like to _catch_ all failed tests, for reporting
to upstream, but I also don't want to block the package just because
the output of one particular solver is slightly different than
expected when run e.g. on ARM or something.)
But you are saying then that if it builds and tests just fine locally
then I should not expect different conditions under ci.debian.net, is
this correct?
Currently I use cowbuilder with `gbp buildpackage` for package
building and schroot for autopkgtest, I don't know if this is optimal
but it seems to work well.
thanks,
Steve
On Thu, Oct 25, 2018 at 12:44 AM Stuart Prescott <stuart@debian.org> wrote:
>
> Hi Steve,
>
> You can run the test suite yourself on your local machine with either
> autopkgtest (using a chroot/schroot/pbuilder/vm/etc) or without isolation
> using sadt from devscripts.
>
> Running the tests yourself only tests against the hardware you have available
> and so isn't probing different architectures, but then ci.debian.net doesn't
> either.
>
> It is possible to build and run tests from gitlab CI. You can write your own
> code for that or see an existing project for it:
>
> https://salsa.debian.org/salsa-ci-team/pipeline
>
> (I'm not sure I like the way it has been implemented in that project but it's
> an option for you)
>
> Gitlab CI is still not probing many architectures.
>
> Established maintainers can get access to porterboxes with various
> architectures to do their own testing and debugging on different hardware. I
> don't know whether that's going to be an option for you, however.
>
> To the source of your problem -- what sort of comparisons are you having
> trouble with? It's generally best that numerical tests use some sort of
> "almost equals" test probing only an acceptable number of significant figures;
> that should then pass on all architectures. Would it help to point to the
> specific code at this stage?
>
> Cheers
> Stuart
>
>
> --
> Stuart Prescott http://www.nanonanonano.net/ stuart@nanonanonano.net
> Debian Developer http://www.debian.org/ stuart@debian.org
> GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
Reply to: