Hi Andreas, On 16-04-2020 10:09, Andreas Tille wrote: >> superficial is translated to neutral. As is FAIL (flaky). > > Hmmmm, what exactly means "superficial". Please read the documentation: https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst > Are all those > Testsuite: autopkgtest-pkg-* > > superficial? No. Only those that only have superficial tests. E.g. ruby runs the full upstream test-suite automatically. > Do they qualify for early testing migration or not? Superficial tests *are* neutral, so no. > Wouldn't it be more informative to have a fourth category > > pass > superficial > neutral > fail No, because there are only three states. I.e. superficial and flaky and skipped tests all end up meaning the same. > My intention was to have a list with packages of our team where either a > test is missing or failing. My idea was that some autopkgtest-pkg-* is > "test is not missing". What is your opinion as debian-ci team about my > idea? It seems you want to be processing the message field then, but honestly if there is an entry, "test is not missing". If there is no entry, "test is missing". Failing is "fail". flaky tests are also "test is not missing", skipped tests are also "test is not missing". Why would you need all those states? You have the "message" field in your UDD schema to check why you get the neutral state. Paul
Attachment:
signature.asc
Description: OpenPGP digital signature