Re: aren't unreliable tests worse than none? (Re: Help requested: Packages which FTBFS randomly)
Holger Levsen <holger@layer-acht.org> writes:
> While I agree with Niels that it would be very worthwhile to be able to
> define ressource requirements for a package to build (and thus know I
> have to life with some packages having trouble sometimes) I find it
> *very* strange to be content with test suites which randomly fail.
The point is that they don't randomly fail in the sense that they don't
fail n% of the time when run in any possible build environment. Rather,
in the subset of cases we're talking about in this thread, the tests work
reliably on the developer's machine and on the buildd network, but fail
either reliably or randomly in other build environments.
This is, in a sense, an unreliable test, but it's not unreliable in a way
that directly affects the main line of package development.
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
Reply to: