Bits from the Release
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi all,
We have some ideas on how to improve the Jessie development cycle.
This includes a proposed change to the testing migration rules,
automated removals, automated metrics for architecture
(re-)qualification and doing a roll call for porters.
Index:
* Changing testing migration (NEW-TEST)
- How you can help (NEW-TEST-HELP)
* Key-packages and automated removals (AUTO-RM)
* Automating Architecture (re-)Qualification (ARCH)
* Roll call for porters (ROLL-CALL)
Changing testing migration (NEW-TEST)
=====================================
Colin Watson from the Ubuntu Release Team suggested some changes to
the testing migration rules[TEST-QUAL]. The basic idea is to replace
the age policy with automated tests (Ubuntu using autopkgtest/DEP-8).
All other criteria remain unaffected (the package still has to compile
and be installable etc.)
At the moment there are unfortunately very few autopkgtest tests. We
therefore propose that packages will be allowed to migrate to testing
*with a delay of 2 days* if and only if:
1. The package has autopkgtests.
2. All of the autopkgtests in the package can be run successfully
(i.e. it is possible to run them and none of them fails).
At the same time, we would like to make autopkgtest failures testing
blockers. Once this infrastructure is in place, we would like to use
it to automatically test reverse dependencies as well. In fact:
3. If the package has reverse dependencies with autopkgtest,
(a random subset of) these tests may be re-run.
4. Any failure in the above-mentioned autopkgtest tests will be
considered a regression and block testing migration. Where
these failures are caused by (misbehaving) reverse dependencies,
bugs should be filed against reverse dependencies.
For this proposal to make sense, all deployed autopkgtests must
actually test the package involved to some extent. We trust it will
not be necessary to establish a technical solution for this part. For
packages without autopkgtests, the existing age policy would continue
to apply.
Furthermore, we hope to have piuparts integration so that severe
install, upgrade and/or removal problems will also block testing
migration. Less severe piuparts issues will be ignored for this
purpose.
We intend for the autopkgtest checks to be performed on at least i386
and amd64. In the future, we may expand this to include other
architectures where hardware permits and "reasonable running time" can
be achieved. We hope this will greatly improve the quality of our
ports as well.
So, in summary: if an upload causes piuparts and autopkgtest failures,
this will block migration. Packages with autopkgtest will migrate as
soon as they have aged 2 days and all tests have been run
successfully.
How you can help (NEW-TEST-HELP)
================================
Add tests to your packages. The full specification for these tests
are available from [AUTOPKG]. If you need inspiration, consider looking
at some of the existing autopkgtests[FIND-AUTOPKG].
We need to have the autopkgtest tests run. Holger Levsen suggested
that jenkins.debian.net has the necessary hardware to support these
tests.
Asheesh Laroia has kindly spent some time during DebConf13 researching
and experimenting with setting up such jobs. Sadly, he does not have
the time to finish it. If you are interested in 2-day migrations for
your packages, we need volunteers to help us implement it. :)
Volunteers for adding and maintaining autopkgtest support on
jenkins.debian.net should contact Asheesh Laroia, Holger Levsen and
us.
Key-packages and automated removals (AUTO-RM)
=============================================
We have been nudged repeatedly at DebConf13 to replace the
semi-automatic removals with fully automatic ones. It was also
suggested to us that the current criteria might need improvement.
At the Release BoF at DebConf13, it was suggested to use (something
based on) Lucas Nussbaum's "key packages" proposal. The original
proposal can be seen at [KEY-PKGS].
Lucas's scripts suggests that the majority of all RC bugs currently
only affect "non-key packages". Out of the about 1300 RC bugs in sid,
"only" ~250 of them affect "key packages". Lucas also integrated "key
packages" support into the UDD bug view[UDD-BUGS].
Lucas is currently working on improving the script finding key
packages. But we would like to request a volunteer to help us rewrite
our tools to automate the removal process. If you are interested in
helping us with this, please contact the release team for more
details. The source code for those tools are available via
[RM-TOOLS].
We will provide more details on how this will work, once we and the
volunteer(s) have a clear idea of how to implement the automated
removals.
Automating Architecture (re-)Qualification (ARCH)
=================================================
Based on feedback from the Release BoF at DebConf13, we will be
looking into automating some (but not all) parts of the Architecture
Qualification.
The general idea is to regularly collect and plot data about all the
ports. This will hopefully help us provide a better overview of how
well the ports are doing over time. We hope these plots will aid us
in making more qualified decisions. However, while the plots may be
used as a basis for making a decision, they will not be used to
automate the decisions themselves.
Besides assisting us, we hope these tools can also aid the porters to
spot and solve potential problems earlier. On a related note, we also
hope to provide a script to help porters find source packages that
hold back their archive coverage.
We would like to thank Ivo De Decker for suggesting some concrete
metrics and working on a prototype to collect those metrics. We would
also like to thank Johannes Schauer for working on a prototype for
finding the "archive coverage"-blockers.
Roll call for porters (ROLL-CALL)
=================================
We are planning to do a roll call for porters of all current ports, to
ensure the porter teams are still sufficiently staffed.
The results of the roll call will be included in our next bits.
Niels, on behalf of the Release Team
[TEST-QUAL]
http://meetings-archive.debian.net/pub/debian-meetings/2013/debconf13/high/1028_Ubuntu_daily_quality_improvements_overlap_with_Debian_CUT.ogv
[AUTOPKG]
http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=blob_plain;f=doc/README.package-tests;hb=HEAD
[FIND-AUTOPKG]
apt-file -a source update
apt-file -a source search debian/tests/control
Examples include Lintian:
http://anonscm.debian.org/gitweb/?p=lintian/lintian.git;a=commitdiff;h=772d35821a0ccbd641a638ccd18476875ce3c5e2;hp=48c5ad8286c6c2cb78eddf47665097631ec4556f
[KEY-PKGS]
https://lists.debian.org/debian-devel/2013/05/msg00496.html
http://udd.debian.org/cgi-bin/key_packages.cgi (NB: This script has a long load time)
source: http://anonscm.debian.org/gitweb/?p=collab-qa/udd.git;a=blob_plain;f=web/cgi-bin/key_packages.cgi;hb=HEAD
NB: In the mail, the "key packages" are referred to as "important
packages". They were later renamed to "key packages" to avoid
confusing them with e.g. "Priority: important" packages.
[UDD-BUGS]
Example: All RC bugs in sid that affects key packages
http://udd.debian.org/bugs.cgi?release=sid&merged=ign&keypackages=only&fnewerval=7&flastmodval=7&rc=1&sortby=id&sorto=asc
[RM-TOOLS]
svn://svn.debian.org/svn/collab-qa/rc-buggy-leaf-package
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
iQIcBAEBCAAGBQJSGhzpAAoJEAVLu599gGRC9koQAJLx9e3agydkppMmSdqp/WNu
OhgUQq+Vffg2kCUNopbuXOcmijSvo3PT8OcG8yBCOA47oJdHY7poT7aAx8X2b5oi
tzrIsuiCAobhUKLUS6ajeieDobUjkVnb8BYF8g0qSVZ0LcBxc62UgAsuBQqfCJoW
BDRQmoyaIYyP0AVDQCxAizhSXAtKXiVG9ZDg8ET7bVrgntmaQ6M9vfDcYnzivedz
YZTrj5ffTOBtqN4d6HAxmXL5rv2jBjChVpPsVFtIVla8/QjyN8bTR0rGzzSd5++l
sMYxbVkb6tK8gBt4y5hbMJmfV7ekqCykRBRy421x27AXawj+m0Zq0oY4X8fh54pM
5NtDewvVeuAync1fqLpgt91iW+BZXvVrRZNxsZBMhw7tyJ4QnJA4TEJKEl470iyC
67ylgCF0Om6ZTFC2i0tEbtbsk89sS7pn7YS3Jv6LyO47Ohqhc5SLSBCaszGTXTp+
xmDWnQEew0Xw+70+C1diYzJp1l8nQYZnSbonAjgjS2VaENu68Rvek05JSExY9Nlt
GCvNDPc2rYRE0Y/sx3UU6ihDpXNVei91UEk0l42hNrIFBvF4FQ/pUr9MRvSxM7R5
8EjnD5RwYSydla95w+H/QvXfdBUJUL5bEMUq/4D5m3xoDxDCHNRCSsBbnhXx4U0r
2plHHe3kOnRgdvAn+mE3
=TDbY
-----END PGP SIGNATURE-----
Reply to: