[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Auto reject if autopkgtest of reverse dependencies fail or cause FTBFS

Paul Wise writes ("Re: Auto reject if autopkgtest of reverse dependencies fail or cause FTBFS"):
> On Wed, Jan 18, 2017 at 1:08 AM, Russ Allbery wrote:
> > Oh, sure, I'm in favor of disabling flaky tests if we can't fix them.  My
> > experience is usually more that I'm leaving them on *because* I'm trying
> > to fix them and can't reproduce locally, or I think I've fixed it (but
> > actually haven't).
> I would expect most upstream test frameworks support marking tests as
> flaky, which usually means they always get run and results printed but
> their outcome never causes a build failure.

It might also be that the flakiness affects all tests.

For example, there is a race in gnupg2's gpg-agent which makes the
dgit test suite fail sometimes.  (#841143, reported in October; now at
last being worked on.)

As it happens, I have chosen (for other reasons[1]) not to run the
test suite during package build, so this is never a FTBFS.  But it
could easily have been a flaky FTBFS.


[1] Mainly, that the test suite is computationally intensive and has
an inconveniently large dependency set; this makes it
disproportionate, particularly given that the actual package build is
almost trivial.

Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Reply to: