Re: Auto reject if autopkgtest of reverse dependencies fail or cause FTBFS
Sean Whitton <spwhitton@spwhitton.name> writes:
> I agree with the principle that test failures should be RC by default.
This is something which seems to have no disagreement here. My concern
is just that I want to have a simple way to override this, to assign
this to a different package etc. I want to have the same flexibility
here as for bugs.
> I think we need an additional field in d/tests/control to mark
> individual tests as non-critical (this wouldn't really help Ole's 9000
> tests package though).
While this is a really large test suite, it also wouldn't help for
others, since there are many packages with > 100 tests: aplpy has ~250
tests, astroml has ~210 tests, gnudatalanguage has ~170 tests etc. Even
when the package itself is rather "small", the test count is quite
nice -- the github ecosystem with test coverage tools (and badges) helps
here, as well as powerful Python test packages.
Especially modern packages with a highly motivated upstream come with a
large number of tests (and they care about failures). This makes
autokgtest to a very valuable tool, but it shouldn't become a
maintainer's nightmare. And if I would have to maintain test-specific
severities, that would be a nightmare, not only for me but also for
upstream, who would need to be involved as the knowledgable instance
here.
Best regards
Ole
Reply to: