Re: [UDD] Is there any information about failed autopkgtest in UDD?
Hi Paul,
On Thu, Apr 16, 2020 at 10:30:17AM +0200, Paul Gevers wrote:
> > Hmmmm, what exactly means "superficial".
>
> Please read the documentation:
> https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst
Thanks for this pointer.
> > Are all those
> > Testsuite: autopkgtest-pkg-*
> >
> > superficial?
>
> No. Only those that only have superficial tests. E.g. ruby runs the full
> upstream test-suite automatically.
Ahhh!
> > Do they qualify for early testing migration or not?
>
> Superficial tests *are* neutral, so no.
>
> > Wouldn't it be more informative to have a fourth category
> >
> > pass
> > superficial
> > neutral
> > fail
>
> No, because there are only three states. I.e. superficial and flaky and
> skipped tests all end up meaning the same.
>
> > My intention was to have a list with packages of our team where either a
> > test is missing or failing. My idea was that some autopkgtest-pkg-* is
> > "test is not missing". What is your opinion as debian-ci team about my
> > idea?
>
> It seems you want to be processing the message field then, but honestly
> if there is an entry, "test is not missing". If there is no entry, "test
> is missing". Failing is "fail". flaky tests are also "test is not
> missing", skipped tests are also "test is not missing". Why would you
> need all those states? You have the "message" field in your UDD schema
> to check why you get the neutral state.
I definitely want to expose the full message (that's why I insisted to
put it into the table). My intention is to expose only those packages
where some work needs to be done. Otherwise the list will be to long.
I will think about the plan after receiving your information.
Kind regards
Andreas.
--
http://fam-tille.de
Reply to: