Hi Andreas, On 15-04-2021 21:13, Andreas Tille wrote: > On Fri, Apr 09, 2021 at 06:32:40PM +0200, Paul Gevers wrote: >>> We are working on it makeing them non-superficial. Its the case for >>> r-bioc-* currently[1] and the plan is to do this for all R packages. >> >> Aha, so non r-bioc-* packages are superficial at this moment. > > Not really. Sorry, then I misunderstood. >>> However, this is for the next release. Marking those packages >>> superficial that do not come with a proper test can be done as well, >>> but not in the current freeze period. >> >> I'll think about what this means for the Release Team. Normally when I >> learn of packages that try to migrate to testing with superficial tests >> not marked as such I would add a manual block. Maybe I'll see if I can >> generate such a list for all autopkgtest-pkg-r using packages (without >> bioc in the name). > > No. The majority of packages r-cran-* packages has "real" tests that > are manually crafted in addition to autopkgtest-pkg-r. Great. Loving it. >> Thanks for letting us know. And yes, please fix this. While typing this, >> I have one suggestion, we should make autopkgtest-pkg-r skippable and >> pkg-r-autopkgtest should exit 77 if there's no hook nor any tests to run >> (and without bioc currently). The overall result of skipped tests is >> equal to successful tests marked as superficial. I believe we >> can/could/should very well do that now in the freeze, after we fix >> autodep8. What do you think? > > I think that's OK since as I said most of the packages have manual > tests. So skipping pkg-r-autopkgtest will not really influence the > tests of the packages in practice. Because you use the word "skipping" here, I think you misunderstood my intentions. What I mean is that pkg-r-autopkgtest is run, but if it determines that it hasn't done any substantial testing, it can exit with exit code 77 which, with the skippable restriction, will result in a NEUTRAL autopkgtest result which is exactly the same as a successful superficial test. Paul
Attachment:
OpenPGP_signature
Description: OpenPGP digital signature