Re: Auto reject if autopkgtest of reverse dependencies fail or cause FTBFS

[Ian Jackson <ijackson@chiark.greenend.org.uk>]

Ole Streicher writes ("Re: Auto reject if autopkgtest of reverse dependencies fail or cause FTBFS"):
> I don't see the need to keep things in sync: If a new failure is
> detected, it creates an RC bug against the migration candidate, with an
> "affects" to the package that failed the test.

I prefer my other suggestion, that humans should write bugs if
necessary to unblock migration.  Because:

 * It eliminates a timing problem, where the testing migration
   infrastructure[1] needs to somehow decide whether the test have
   been run.  (This is needed because in the future we may want to
   accelerate migration, perhaps dramatically when there are lots of
   tests; and then, the testing queue may be longer than the minimum
   migration delay.)

 * See my other mail about the problems I anticipate with
   automatically opened bug reports.

> > Than maybe change the wording: not-blocking, for-info, too-sensitive,
> > ignore or ....
> 
> The problem is that a test suite is not that homogenious, and often one
> doesn't knows that ahead. For example, the summary of one of my packages
> (python-astropy) has almost 9000 individual tests. Some of them are
> critical and influence the behaviour of the whole package, but others
> are for a small subsystem an/or a very special case. I have no
> documentation of the importance of each individual test; this I decide
> on when I see a failure (in cooperation with upstream). But more: these
> 9000 tests are combined into *one* autopkgtest result. What should I put
> there?

You should help enhance autopkgtest so that a single test script can
report results of multiple test.  This will involve some new protocol
for those test scripts.

Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.