Re: Are we ready to block on autopkgtest regressions?
Paul Gevers writes ("Are we ready to block on autopkgtest regressions?"):
> Three days ago I opened a merge request against brintey2 [1] to
> enable britney2 to block migrations that cause regressions in
> autopkgtest results in testing. Niels copied it to the IRC channel,
> but we saw no reactions so far from other RT members. We were
> wondering what the opinions in the team are: should we go for it?
> And if not, what anre the issues that you consider to be in need of
> fixing first.
Thanks for pushing this. I do want to see the autopkgtest influence
on migration increase. But:
I think you may be underestimating the degree to which this will need
social as well as technical adjustments. In particular, I don't think
the current increased migration delay approach has fully tested what
will happen when autopkgtest-detected problems actually start to get
seriously in people's way.
Right now if your package migration trips an autopkgtest regression,
this is still not really your problem. You can sit on your hands and
wait for your rdependency's maintainer to swing into action - which
they have to do very promptly. The extra migration delay might be
mildly irritating but the problem will go away before you work up the
energy to argue with anyone about it.
And if you think the problem is a broken test in your rdependency,
there little no point doing more than pointing this out. For example,
a possible response would be to NMU the rdependency. But if one were
to do that according to the existing NMU guidelines, it wouldn't make
much difference to one's own package.
Gradually increasing the extra migration delay would allow us to
gradually increase the proportion of maintainers for whom the
autopkgtest blocking is soemthing they might be motivated to escalate.
Not only would this give us time to further improve the technical
arrangements, but it would allow us to mentor and assist maintainers
unfamilair with the processes, and develop some best practices and
social norms, on a somewhat smaller scale.
I think it is important to work on these aspects before the release
freeze combines with autopkgtest regressions to gore people's oxes
in a higher stakes scenario. So certainly we should be thinking about
this now.
At the risk of overcomplicating things, I suggest replacing the
existing fixed delay with a formula which adds one or two days of
additional migration delay per week of elapsed time, or some such.
Also thought should be given to what `urgency=high' should do
Particuarly, given the bugs that mean it is sometimes specified by
mistake.
Ian.
Reply to: