Debian QA Policy Draft
Below is the document the people who cared about QA agreed upon some time
ago. Since the QA team has not been very active (at least not as much as I
expected) it never had been really followed.
Do people still agree on it's content?
I also post it here so people who are new ( < 1 year ) to the project can
read it.
Note that it's still a draft.
Comments? Feedback?
========================================================================
-- Debian QA Policy Draft --
<Vincent.Renardias@waw.com>
Feb. 21th 1997
Aim:
----
Try to improve the overall quality of the Debian GNU/Linux distribution.
Facts:
------
- Some packages appear to be orphaned or not maintained enough.
- Some outstanding and easy to fix bugs are still pending after months.
- There's a need to check continuously the distribution's stability/quality.
- Need to have a spool of maintainers to take care of orphaned packages.
Foreword:
---------
Sofware QA rules already exist (ISO 9000, etc, ...), but I think they're not really
adapted to Debian, since unpayed volunteers can't accept the same "pressure" than corporate
developpers. Moreover, developping for Debian must remain fun (and not become a pain because
of a QA group barking at every developper ;). Also, the DQAG (Debian Quality Assurance Group)
should not reduce too much the development speed of Debian.
For now, we'll use some 'soft' QA rules, just to make sure the minimum expected
amount of quality is reached and critical bugs are fixed. If everything goes well
and the maintainers are happy with the work the the DQAG,
then more restrictive rules will be used.
Since we do not know yet how efficient/inefficient the DQAG work will be (and how it will
be welcomed by the maintainers), I propose to set it up in 3 steps:
- 1st step will be applied as soon as the DQAG work begins.
- 2nd step will start after 2 or 3 months, when:
- the DQAG will have enough volunteers.
- maintainers will be used to work in collaboration with the DQAG.
- 3rd step will eventually be applied, after discussion/agreement on how to implement it.
Maintainers:
------------
How much work is expected from the maintainers?
- Fix security bugs within 1 week.
- Fix the 'cosmetic' bugs within 1 month.
- Upgrade a package to the new upstream version within 2 months.
- *Try* to fix the other bugs within 3 months.
- Email the QDAG if they 'give up' on a bug.
[ Does the above seems reasonable, or is it {too much,not enough} ? ]
DQAG Evolution:
===============
* 1st step: (Effective as soon as this draft becomes the official policy)
-----------
- Immediate work:
* Re-upload the known orphaned packages with the 'Maintainer' field set to:
"Orphaned Package <debian-QA@lists.debian.org>"
* Send an email to all the developpers listed in 'indices/Maintainers' and ask them
if they're still maintaining their packages (Already proposed as a "developper ping").
[ NB: After looking at this file, I see that _many_ developpers use several different emails
for their packages (one developper is even listed with 4 different emails). If think it would
be a good thing to ask maintainers to use only _one_ email. (Not necessarily their
address <xxx@debian.org>, but always the same address!) ]
* Set up a 'debian-QA' mailing-list where QA issues will be discussed.
- General rules:
* DQAG members:
Every Debian maintainer/user is encouraged to subscribe to the DQAG mailing list
and become an active DQAG member.
* Bugfix upload policy:
* Maintained packages: Send fixes to their maintainers, and eventually
upload the fixed version if no feedback from the maintainer.
(see "uploads delays" below)
* Orphaned packages:
Bugfix uploads can be done any time by DQAG members.
Call for a maintainer on debian-devel;
If no-one found after 1 month:
- Important packages (ie: "required" or "standard"):
The package will be maintained by DQAG until a maintainer is found.
- Unimportant packages:
If there's still no-one volunteering to take care of them
after this delay, they'll be dropped in the section 'project/orphaned'.
* Takeover uploads delays:
After a patch is posted by the DQAG to a maintainer, how long
will the DQAG wait before uploading the package if the maintainer does
not do the upload himself?
Delays :
- critical security fix: 2 days.
- security fix: 7 days.
- bug fix: 30 days.
(Fix a bug reported in the bug tracking system or RFE)
- cosmetic fix: 45 days.
(typo. in man page, core file in source package,...)
NB: "cosmetic fixes" are uploaded to "unstable" only; (neither "stable" nor "frozen").
The above delays are reduced by a factor of 2 in the month preceding a freeze.
During a freeze: Delay for security fixes (those announced on <security@debian.org>): 1 day.
Other fixes: 1 week.
[Are these delays too long/too short?]
DQAG members making bugfix uploads will need not only to respect the delays above
but will also have to announce their intent to upload on <debian-QA> with a cc: to the
maintainer at least 2 days (or 1 day
during a freeze) before to do the upload, saying which Package_Version they will upload,
and the serial numbers of the bugs fixed by the upload.
The 'Maintainer' field of those uploaded packages will stay unchanged.
However if DQAG members make 3 consecutive bugfix uploads with still no action
from the package maintainer, then the 'Maintainer' field will be set to
"Orphaned Package <debian-QA@lists.debian.org>", and the package will be considered as
orphaned (this will be announced on <debian-devel>).
- work on "stable" and "unstable" distributions:
* Check distribution completness (wrt dependancies) after each dinstall run.
(for stable and unstable, with and without contrib & non-free, and for each architecture)
* Check bogus/weird/outdated dependancies:
- packages depending on (say) libc_5.0.9 (need to recompile to check if it still
works with libc.5.4 (and soon libc.6))
- package depending on something that apparently has nothing to do with it.
* Scan the bug tracking system for bugs older than 3 months that are fixable and provide
patches to the debian maintainers (and eventually also to the upstream maintainers).
* "repackaging/updating work": eg:
- new package format.
- patch packages to use bash-2.0/shadow/locales/glibc/PAM/...
- Test of frozen distributions:
[ To be coordinated with "Test Manager" Dale Scheetz ]
* As soon as a distribution is frozen, the 'testers' will try to make an install/upgrade
with the frozen distribution.
* 2 sets of tests:
* Install "from scratch".
* Upgrade from previous "stable" version.
After a tester finished an installation/upgrade, he must send a report to Dale Scheetz saying:
-----------------------------------
* the hardware used (CD-ROM type, SCSI card, etc).
* if trying to do an upgrade (from Debian-1.1? Debian-1.2?) or install.
* the install method (FTP, NFS, CD-ROM, ...)
* Any other relevant informations.
If problem(s) found:
* what went wrong with as many details as possible
(including the exact error messages whenever possible),
* the possible cause of the bugs,
* the fix(es) he eventually suggests to fix the problem(s).
if no problem found :
* say "Everything went fine".
-----------------------------------
Bugs will also have to be sent to the debian bug tracking system by the tester if the
package(s) causing the bug(s) can be clearly identified.
If at least 2 testers report the same installation/upgrade problem, this bug will be marked as
critical and will have to be fixed before the frozen distribution can be released.
* 2nd step:
-----------
step 1 +
* One current problem with Debian, is that there's only one developper assigned by package.
[ This part will be done with Dale Scheetz too? ]
In a near future, I propose to have a pair of developpers by package:
- One doing the 'real work' (ie: what maintainers already do by now with their
packages)
- The other one checking each package release for bad dependancies, missing man
pages, ... If a problem is found a bug report must be opened, and whenever possible
the tester should try to include a patch to fix the reported bug.
(Since testers aren't supposed to make uploads, they don't necessarily need to be
"Debian developpers" having an account on master.)
For {big,important,bug prone} packages (ie: emacs,bootdisks,gcc,libc6,...),
there may be several testers.
* Setup some 'test suites' (and run them periodicaly) ?
- for the "complete distrib"? (eg: POSIX compliance)
- for a functionality (eg: java support, TeX).
- for a precise package: (eg: 'compiler self test' for guavac, bison, ...)
* Scan c.o.l.a. & sunsite's new entries, and open bugs on concerned packages asking
for upstream upgrade. (Yet another way to see if maintainers take care of their packages ;)
* 3rd step?:
------------
step 2 +
* Have an "official group" of testers, who will systematicaly not only test *every* package
released into "unstable" before to put them in the "tested" distribution, but may also REFUSE
them into this distribution in case of critical bug(s).
------------------------------------------
* Open problems:
[ Feel free to comment ]
- How to make sure a maintainer doesn't "overwrite" a fix made by the DQAG?
[ Ask the maintainer to acknowledge the DQAG fix by email? ]
- Who will make the final decision when the DQAG and the package maintainer disagree
with each other on a bug fix?
(1/ the DQAG, 2/ The maintainer, 3/ The BoD, 4/ Someone else?)
- Implement Lars' idea of a 'tested' distribution (3rd step).
NB: The idea is very good IMHO, but I have no idea of how to implement it
without too much overhead and not stepping on too many feet (see introduction).
Needs more discution about it.
---
IMPORTANT NOTES:
* The proposed fixes must be taken into account by package maintainers!
* There already are too many bugs which are outstanding since months
and are fixable with a 2 line fix provided in the bug report!
* The DQAG must help maintainers, but does NOT replace them.
* Also, maintainers must not rely on the DQAG to take care of their packages, or fix their
bugs...
* The DQAG is only meant to (in no particular order):
- take care of orphaned packages.
- HELP maintainers by providing fixes they should apply THEMSELVES.
- detect as many bugs as possible (and as soon as possible) in the uploaded packages.
------------------------------------------------------------------------------------------
--
- Vincent RENARDIAS vincent@{waw.com,pipo.com,debian.org} -
- Debian/GNU Linux: Pipo: WAW: -
- http://www.fr.debian.org http://www.pipo.com http://www.waw.com -
---------------------------------------------------------------------------
- "La fonctionnalite Son Visuel vous delivre des avertissements visuels." -
- [Message durant l'installation de Windows95] :wq
--
TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word "unsubscribe" to
debian-devel-request@lists.debian.org .
Trouble? e-mail to templin@bucknell.edu .
Reply to: