RE: Comments on Debian Quality Assurance (long)
Hi,
this is quite long, since there are so many related
issues here. I try to touch on some, but not all are even
mentioned.
What is quality in Debian?
==========================
When discussing quality of Debian three separate but
interconnected issues should be addressed:
A) quality of software in Debian (i.e. quality of software
that is being packaged for Debian).
B) quality of packaging of that software (i.e. quality of
work that is being done when adapting the software to
Debian).
C) quality of Debian system (i.e. selection of suitable
packages and defining how they should be configured to
make an integrated whole).
Often these issues are confused, which makes discussion
difficult at times.
How these issues are handled:
A) is easiest to define, since Debian is primarily a packaging
organization. Probably less than 5% of packages are Debian
specific. Quality of software is dependent on upstream
quality. Debian supports it by Bug Tracking System and by
forwarding any bugs, fixes and enhancements upstream.
Some maintainers do more but that is not required. Also doing
more is conceptually difficult, since having a version that
is modified from upstream is considered forking by many.
Forking is considered inherently bad for free software.
B) is mainly handled by Debian Policy document and policy
conformance checking tool Lintian. Policy is mainly discussed
on list 'debian-policy'. The other main document is the
Debian Developer's Reference. There are some other documents
available in Developers Corner. There are also several tools
to partially automate package creation and maintenance. These
are designed to eliminate some of the most common errors.
C) is also handled by Policy and various subpolicies (e.g. Perl
policy for Perl packages, emacsen policy, etc.)
Technical issues related to all of A) to C) are often brought
up on all mailing lists and discussed in depth, often with great
expertise.
Quality assurance in voluntary organization
===========================================
Modern quality thinking seems to place much emphasis on planned
quality and then verifying that planned quality has been achieved.
This is a difficult one for a large voluntary project like Debian.
Here decisions are made mostly by discussion and consensus, not as
managerial decisions. Also there is no practical way to control
the work of an individual as it is being done. By necessity all
controls are either before the work or after the work. Also the
pre-work controls are weak by nature, there is no way of preventing
anybody doing any work (in any way they want to do it), the only
real control is when incorporating that into Debian. On the other
hand, even weak controls are very effective on the long run,
since the main rewards are peer recognition, which cannot be
achieved with poor quality work, and work satisfaction, ditto.
This means that one of the most important issues is induction of
new developers. This has recently been improved, but could still
be better. There is also the list 'debian-mentors' for asking any
packaging related question or requesting private tutoring on
packaging. Other important issue is keeping maintainers active
for long time. This has been recognized but no solution has been
found yet.
There are almost no enforced reviews or strictly enforced check
points, but there is a very strong culture of peer reviews. Even
though change management and configuration control are rather
informal I still consider the achieved result to be at least as
good as in many commercial organizations. This is mainly due to
very strong commitment to quality in Debian and community effort
in doing the work.
There has been several times discussion on 'debian-test' list about
testing framework and requiring developers to supply some
regression tests or at least list of functionality to test, but
this has not yet led to any concrete requirements for developers.
There is an open policy proposal, however.
Defined processes for quality
=============================
Debian tries to be *very anti-bureaucratic*, but some processes
have been defined. Most important for quality are probably these:
- accepting and purging maintainers
these try to ensure commitment to free software, not any
skill set or experience level, motivation is considered
proven by application to voluntary organization
- adopting and orphaning packages
if there are more than one applicant then previous
maintainer or other respected person selects; if maintainer
cannot maintain the package he eventually orphans package
- non-maintainer uploads
for critical bugs or packages that are not maintained
- uploading packages
includes authentication, quality control and bug handling
- making releases and point releases of Debian distribution
stable release, freeze, release critical issues,
distribution testing
- bug handling
recommended response times, proper bug closing
- resolving technical disputes
discussion on relevant list, if no consensus then resolution by
technical committee or by general vote
Other resources
===============
There is no single list or web site for QA (in the modern sense)
but here are some of the most relevant ones:
Debian QA group:
mailing list 'debian-qa'
web sites:
http://qa.debian.org/
http://tux.u-strasbg.fr/~raphael/debian/qa.html
Debian Test group
list 'debian-testing' in the 'Other' section of the mailing
lists page.
Debian Policy group
list 'debian-policy'
proposals are tracked in Bug Tracking System
Closing comments
================
Perhaps you would briefly summarize the difference in QA and QC.
There are different wievs on that, and it would be nice to know
your standpoint.
Please note, that almost all of the activities (check ...) are
actually guidelines for selecting where the improvement work
should be directed, not what I'd call pure quality control.
As you have noticed, there is no QA reference model, but if
somebody (you?) is willing to take the initiative he will
certainly receive support from many smart, motivated and
knowledgeable persons.
There is no (and probably cannot be any) design control at package
level, but there has been quite a lot of effort at distribution
level. Mostly that is codified in Policy and Developers Reference.
Even if the title Quality Assurance is not totally accurate it
probably should be preserved for historical reasons. Also
activities can always be extended in the right direction, if
volunteers are available.
Disclaimer:
I have been hanging around and reading many
lists since '95, but I'm not a developer, so treat this as
an informed opinion but without any authority.
t.aa
Giovanni Biscuolo <teknofin@libero.it> Wed Nov 10, 1999 3:22 PM
>
> I'm reading your page (http://www.debian.org/distrib/) about
> QA in Debian and was very
> excited reading the title Quality Assurance for I am an
> enthusiast of Quality culture. I think it's a great thing
> for GNU Software.
>
> Anyway, I would like to point out that all the activities you
> mention there (check ...) are about Quality Control, not
> Assurance. It's not my intention to teach you the difference
> - I have nothing to teach - but the difference is big.
> I'm very interested, for example, to know your QA reference
> model (ISO 9001 i.e.), how do you plan design control and,
> why not, if you are going to develop
> and publish a GNU/Debian QA Manual.
> I think your WWW page should specify deeper the activities
> and documents about Quality Assurance
> or you should change the title to "Quality Control".
>
> Last but not least: is there a mailing list about this topic?
Reply to: