Quality Assurance in Debian
Hi,
I didn't want to announce this today but since Joey has brought up
a hot topic on -devel this morning, I feel that it's time
to make public what some people (me, Christian Kurz, Tortsten Landschoff,
Josip Rodin, Vincent Renardias and Martin "Joey" Schulze) have
prepared for resurrecting debian-qa.
Please read the text attached (or the html version you can find here :
http://tux.u-strasbg.fr/~raphael/qa/doc/qa.html/). Any comments are
welcome.
[ Please read below only once you read the text mentionned in the
paragraph above ]
I believe that the proposed organization (associated with the tools
developed aka 2 mailbots, a database and the website) can be the
solution for :
- following the Release Goals. If you wonder why I said that check out
http://tux.u-strasbg.fr/~raphael/qa/big-tasks.html
Each of the release-goal can be splitted in simple tasks. Everybody can
follow what needs to be done on the web and can help.
It's inspired from the Gnome Status Page.
- correcting buggy packages [more explication below why it could work]
- finding "cruft" and maintainers that have vanished (this is the
job of the QA committee)
Now some comments why this can work. Many people expressed their interest
to work for Debian's quality but they said it was difficult because :
- they don't know where to begin
- they don't want to work too much (they'd like to grab a little task they
can do in 2-3 hours and after that forgot QA since the following and so
on)
- they don't want to continue writing patch if they are not used. Sometimes
patches sent to the BTS are simply ignored (which does often mean that
the maintainer vanished)
- they don't want to do NMU because they don't want to argue with
the maintainer
- even if they want to do a NMU, they must contact the maintainer
and wait for his response. If the maintainer doesn't respond, they
should wait 3 weeks (it's 3 times enough to forget about the
NMU they wanted to do) ...
Now, let's listen how those problem are solved :
- they don't need to choose a package and a bug to correct, they simply
work on what the committe has brought up to them
- the committee has splitted the work in many little task so that everyone
can help
- once someone has provided a patch for the task, it is marcked as "patch
available". Those tasks are stored in a special list that the
committe check from time to time. When a task is marked as patched for
more than 3 weeks the committee can ask for an NMU in order to
integrate the work. Note that the person doing the NMU don't have to be
the person who wrote the patch.
- people doing the NMU at the request of the QA committee don't have to
fear the maintainer's reaction. They'll simply respond "I did it
because the QA committee decided it, check with them if you have a
problem".
The main tool for debian-qa will be a new (dynamic) web site where
the tasks to do are listed. The tasks where a patch is available
are also listed separately. And last but not least you can find a list
of meta-tasks. The meta-tasks are big tasks composed of many little
tasks. This is a practical way to follow the release goals and other big
jobs like "Killing release critical bugs" and so on.
The web site can be easily updated by sending mails to 2 mailbots. The
committee only can create new tasks but everybody can update the
information available for each task (setting the task as 'patch
available', closing it, setting the status field, ...).
What can I say more ?
I hope that we can agree on this proposal (i've already spent many hours to
write the tools) and that we can start to try this way of working.
Oh yes, for the QA committee: myself, Christian Kurz and Tortsten
Landschoff are volunteers to begin.
BTW, do we need to vote the proposed text ?
One of the main thing I want to do right at the beginning of our work is
contact all maintainers and ask them to reply (yes a ping message where
you have to reply to explicitely tell that you are still with Debian). They
will be also asked if they don't want to orphan some of their packages,
that they do no more use.
Maintainer that do not respond could be removed from the keyring later
(some months after another try). And the packages orphaned that nobody
wants to adopt could be removed (yes there are some unuseful packages,
like psptools).
Cheers,
--
Raphaël Hertzog -=- http://tux.u-strasbg.fr/~raphael/
<pub> CDs Debian : http://tux.u-strasbg.fr/~raphael/debian/#cd </pub>
Quality Assurance in Debian
---------------------------
Raphaël Hertzog <hertzog@debian.org>
Christian Kurz <shorty@debian.org>
0.2
-------------------------------------------------------------------------------
Abstract
--------
With the growing number of developers working on Debian (>500
registered developers at the moment), the increasing number of
packages (>3500 already) and the large number of bugs (about 10000)
Quality Assurance (QA) is urgently needed. This document explains a
proposed solution for Debian about the quality assurance issue.
Copyright Notice
----------------
Copyright 1999 by the Debian Project
This text is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version.
This is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
A copy of the GNU General Public License is available as
/usr/share/common-licenses/GPL in the Debian GNU/Linux distribution.
-------------------------------------------------------------------------------
Contents
--------
1. Why do we need Quality Assurance ?
1.1. Some facts
1.2. Goal of the proposed organization
2. Overview
3. The different entities
3.1. The QA committee
3.2. The QA core team
3.3. The QA members
-------------------------------------------------------------------------------
1. Why do we need Quality Assurance ?
-------------------------------------
1.1. Some facts
---------------
* The number of bugs is growing as is the number of packages.
* Some packages have many bugs because the maintainer left Debian
without orphaning his packages.
* Some maintainers can't manage all their packages and accept to
live with buggy packages.
* Some bugs are sometimes too difficult to correct for the
maintainer.
* There's also period during which some maintainers can't work at
all for Debian, but their packages may still need some attention.
* In general, the quality of Debian can be improved in various way.
There are still lots of packages not using all the suggested
tools and integration utilities (menu files, doc-base files,
alternatives, proper section/priorities, ...).
1.2. Goal of the proposed organization
--------------------------------------
Working on Quality Assurance is a big job, in fact it should not be
the job of some volunteers only. Each Debian developer should take
care of the quality of his packages. However this cannot be enforced
by any rules because quality is sometimes a point of view. Therefore
this document won't define strict rules for the packages (the Debian
policy already dictates the rules) but a global organization so that
every volunteer can help by doing things that will concretely improve
the quality of Debian. One of the main point here is scalability, many
people must be able to work together on quality assurance without
interfering too much.
-------------------------------------------------------------------------------
2. Overview
-----------
The work is coordinated on <debian-qa@lists.debian.org>, there's an
official committee which is in charge of leading the volunteers that
want to help with QA issues. There's also a QA core team (composed of
regular QA workers) that can do NMU when the committee request it.
The QA committe can lead people into different tasks :
* provide help/patches for particular bug reports (reducing the
number of bugs in buggy packages).
* check if release critical bugs are corrected rapidly, if after 2
weeks one is not corrected, they may contact the maintainer and
ask him if he wants help or NMU ...
* help for mass update, maintain list of updated packages, send
mail to the maintainers, allow NMU after X weeks without
response.
* check if all packages providing user programs (X11 or not) come
with menu files.
* check that all packages that provide documentation register it
with doc-base.
* check if package are policy conforming (lintian is very useful
for this).
* provide more documentation where it's lacking.
* check if old packages (ie without recent upload) still work and
if they are still maintained.
* check if the informations provided by the packages are accurate
(dependencies, sections, priorities, description, ...).
* check if all the packages are linked against the proper libraries
(so that old libraries may be dropped in favour of most recent
ones).
* check that we have the latest upstream version.
* any other useful (QA related) job.
-------------------------------------------------------------------------------
3. The different entities
-------------------------
3.1. The QA committee
---------------------
It's composed of volunteers designated by the leader. Their members
would manage a database of QA related tasks. They lead the QA core
team and the QA members. They contact maintainers which could benefit
from some help because they have buggy (or difficult) packages. They
offer the assistance of the QA members. But when the QA group has
provided help (via patches in the BTS in general), the maintainer has
to upload corrected packages (considering of course that the patch are
ok according to him) in a time frame of 3 weeks (or 1 week during a
freeze). After this delay, the QA committee can ask the QA core team
to NMU the package in order to integrate the patches and to correct
all the bugs that can be corrected.
If the maintainer doesn't respond to the mails from the QA committee
in a delay of a month (considering that the maintainer did not
announce that he's in vacation), the committee may first allow a NMU
to be done to correct the bugs and may later (two months after the
initial mail) orphan the package if he still hasn't responded.
The QA committee is also in charge of the WNPP (Work Needed and
Prospective Packages) list since they are already contacting
maintainers about packages they might want to orphan.
3.2. The QA core team
---------------------
It's composed of QA members designated by the QA committee, only
people actively involved in QA for more than a month can become member
of the QA core team. Beeing member of the QA core team doesn't give
you many power, the reason of the existence of the QA core team is to
know who can do NMU when the committee request it. However you can be
proud to be part of it, because this does mean that you're doing a
great work for Debian and his quality. If a member of the QA core team
suddenly stops working for QA, he will be removed from the list.
However he can rapidly integrate it again when he restarts working for
QA.
3.3. The QA members
-------------------
Everybody who is subscribed to <debian-qa@lists.debian.org> is a QA
member. The more QA members the better ... the task list will be
posted to debian-qa by the committe once or twice a week, the QA
member are encouraged to work on the problems submitted by the
committee. A QA member who has worked a lot in this area can ask the
committee to be added to the list of QA core team members.
-------------------------------------------------------------------------------
Quality Assurance in Debian
Raphaël Hertzog <hertzog@debian.org>
Christian Kurz <shorty@debian.org>
0.2
Reply to: