[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [UDD] Is there any information about failed autopkgtest in UDD?

Hi Andreas,

On 16-04-2020 10:09, Andreas Tille wrote:
>> superficial is translated to neutral. As is FAIL (flaky).
> Hmmmm, what exactly means "superficial".

Please read the documentation:

> Are all those
>    Testsuite: autopkgtest-pkg-* 
> superficial?

No. Only those that only have superficial tests. E.g. ruby runs the full
upstream test-suite automatically.

> Do they qualify for early testing migration or not?

Superficial tests *are* neutral, so no.

> Wouldn't it be more informative to have a fourth category
>    pass
>    superficial
>    neutral
>    fail

No, because there are only three states. I.e. superficial and flaky and
skipped tests all end up meaning the same.

> My intention was to have a list with packages of our team where either a
> test is missing or failing.  My idea was that some autopkgtest-pkg-* is
> "test is not missing".  What is your opinion as debian-ci team about my
> idea?

It seems you want to be processing the message field then, but honestly
if there is an entry, "test is not missing". If there is no entry, "test
is missing". Failing is "fail". flaky tests are also "test is not
missing", skipped tests are also "test is not missing". Why would you
need all those states? You have the "message" field in your UDD schema
to check why you get the neutral state.


Attachment: signature.asc
Description: OpenPGP digital signature

Reply to: