QA expectations (Was: Do we want to Require or Recommend DH)
Let me briefly hijack the discussion for a side note. ;)
On Tue, May 14, 2019 at 11:30:11AM +0200, Andreas Tille wrote:
> NMUers should do debdiff - no matter what change was done. And yes, it
> happened also to me in the past once or twice that I uploaded an empty
> package or package missing some files. I do not remember whether it was
> connected to a change to dh or not ... but mistakes happen. The fact
> that we might end up with broken NMUs is IMHO orthogonal to any cdbs to
> dh change.
After a failure, we tend to bash uploaders:
* Why didn't they look at the debdiff?
* Why didn't they run piuparts?
* Why didn't they run lintian?
* Why didn't they run autopkgtest?
* Why didn't they do an arch-only/indep-only build?
* Why didn't they test their package?
* ...
The things you have to remember before doing an upload are insane.
Having humans remember all this crap is not a reasonable expectation. I
think our upload process is a bit like classical debhelper: You remember
to do all the things. We've seen the argument that the dh sequencer
sheds light on the unusual aspects of a package. I argue that this
should apply to QA as well. People shouldn't have to remember all the
QA. QA should just work and QA should tell people about the (unusual)
failures.
Now one can turn this argument upside down. One can say: unstable is the
QA area. Britney prevents testing migration on autopkgtest/piuparts/
missing binaries. In that case, we should simply stop filing such things
in the BTS and stop doing manual QA on unstable. It should be ok to
break unstable. But this is not going to work with transitions. Thus I
still think we're doing it wrong and unstable isn't the place to do the
QA we expect from everyone.
Now I wonder how to move forward with this (after the dh discussion).
Helmut
Reply to: