[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [UDD] Is there any information about failed autopkgtest in UDD?

Hi Paul,

On Thu, Apr 16, 2020 at 10:30:17AM +0200, Paul Gevers wrote:
> > Hmmmm, what exactly means "superficial".
> Please read the documentation:
> https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst

Thanks for this pointer.
> > Are all those
> >    Testsuite: autopkgtest-pkg-* 
> > 
> > superficial?
> No. Only those that only have superficial tests. E.g. ruby runs the full
> upstream test-suite automatically.


> > Do they qualify for early testing migration or not?
> Superficial tests *are* neutral, so no.
> > Wouldn't it be more informative to have a fourth category
> > 
> >    pass
> >    superficial
> >    neutral
> >    fail
> No, because there are only three states. I.e. superficial and flaky and
> skipped tests all end up meaning the same.
> > My intention was to have a list with packages of our team where either a
> > test is missing or failing.  My idea was that some autopkgtest-pkg-* is
> > "test is not missing".  What is your opinion as debian-ci team about my
> > idea?
> It seems you want to be processing the message field then, but honestly
> if there is an entry, "test is not missing". If there is no entry, "test
> is missing". Failing is "fail". flaky tests are also "test is not
> missing", skipped tests are also "test is not missing". Why would you
> need all those states? You have the "message" field in your UDD schema
> to check why you get the neutral state.

I definitely want to expose the full message (that's why I insisted to
put it into the table).  My intention is to expose only those packages
where some work needs to be done.  Otherwise the list will be to long.
I will think about the plan after receiving your information.

Kind regards



Reply to: