[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#866878: marked as done (release.debian.org: should autopkgtest regressions be considered RC?)

Your message dated Thu, 7 Mar 2019 20:25:16 +0100
with message-id <001ed52f-41b7-4d58-b91a-4ff7e3cc1be1@debian.org>
and subject line Re: Bug#866878: release.debian.org: should autopkgtest regressions be considered RC?
has caused the Debian Bug report #866878,
regarding release.debian.org: should autopkgtest regressions be considered RC?
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
866878: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=866878
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: release.debian.org
Severity: wishlist

Dear Release Team

I recall some discussion from dc16 and I see a talk planned for dc17
[1] on using autopkgtest results for unstable to testing migration.
In other words, package uploads that cause autopkgtests to fail, where
those tests passed previously, should be prevented from migrating to
testing.  The regression can be in the uploaded package's own
autopkgtests, or in those of a reverse dependency.

Can a decision please be made, as to whether autopkgtest regressions
will be considered RC for buster, so that bugs of severity level
'serious' can be filed now?

Regards
Graham


[1] https://debconf17.debconf.org/talks/2/

--- End Message ---
--- Begin Message --- 
On Mon, 15 Oct 2018 18:41:00 +0000 Niels Thykier <niels@thykier.net> wrote:

> An update on this:
>
> Per
> https://lists.debian.org/debian-devel-announce/2018/09/msg00004.html,
> autopkgtest regressions will block migration to testing before (or no
> later than) the transition freeze.  Due to the slowly increasing
> delay, the regressions will effectively be RC several weeks before the
> transition freeze.

We are now blocking on regressions, so we are considering autopkgtest
regressions RC in principle (for the package of the version in unstable
that causes the regression). However, there are subtleties that need to
be taken into account:

1) regressions caused by too sensitive tests (e.g. failing on
deprecation warnings) or tests that just need to be updated to updated
behavior (e.g. tests that record stdout or stderr and check verbatim
against a reference) are not RC for the package causing the regression.
But in general we prefer those tests to be updated before the causing
package migrates.

2) when autopkgtest starts to regress due to a bug *fix*, i.e. where the
tested package relies on the former, buggy, behavior, it depends on the
situation what needs to be fixed. Normally the tested package should be
updated (and given reasonable time to do so), but of course there are
exceptions to that rule. Of course, if it's not the package, but only
the autopkgtest that relies on the buggy behavior, its the autopkgtest
that needs to be fixed.

3) the Debian setup to test for regressions naturally has limitations.
The setup may not be blocking the package that really causes the
regression. E.g. the test may be installing other packages (direct or
indirect (test) dependencies) from unstable that are really responsible
for the regression. The package will be blocked from migration, but the
RC issue is not against the package.

4) once a regression "migrates" to testing, there is no regression
anymore per definition. So even when a package causes another package's
autopkgtest to fail where it succeeded in the past, the mere fact that
the autopkgtest fails is not RC. Maybe (some time after the release of
buster) failing autopkgtest (in testing) will be an RC bug on its own,
but not yet.

5) ... [I don't pretend being complete in my listing of subtleties.]

Of course, in order to have the package migrate to testing that causes
or seems to cause the regression, manual action is required.

Paul

Attachment: signature.asc
Description: OpenPGP digital signature


--- End Message ---
Reply to: