Re: Let autopkgtests be gating for testing migration in Buster: heads-up and brain-dump

[Martin Pitt <mpitt@debian.org>]

Hello all,

Julien Cristau [2016-12-31 17:45 +0100]:
> > > 2b) let britney generate a list of tests it would like to perform
> > Is 2b really necessary?  I'm not sure why we would need that.
> > 
> Or to be more correct, I'm not sure we would *want* britney to be able
> to (or have to) do that.

I find britney the best place for it both 

  conceptually: it's the piece that makes policy decisions and has overrides
  for broken/ignored tests

as well as

  operationally: it already has knowledge which package groups want to
  propagate and has an efficient way to compute reverse dependencies and other
  information such as Testsuite-Triggers: from source packages; and it's the
  place which has to interpret these results, so it must know which ones to
  look for anyway.

It's IMHO not the place of the test execution engine to prescribe which tests
to run, as that limits its utility to just one use case (and we use Ubuntu's
engine for lots of other cases that just britney testing migration) and
duplicates/potentially disagrees with logic and decisions that britney already
makes.

TBH I'm a bit surprised by your remark, I would have thought it was quite
obvious that this should be under the release team's/britney's control -- but
apparently not then. Where would you like this policy/decisions to live?

Thanks, and happy new year!

Martin
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)