[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

[arto.astala@nokia.com: RE: Comments on Debian Quality Assurance (long)]


[I'm forwarding a few things to this list which I think should be read
afresh from the viewpoint of the technical committee.]



----- Forwarded message from arto.astala@nokia.com -----

From: arto.astala@nokia.com
To: teknofin@libero.it, vincent@waw.com, debian-qa@lists.debian.org
Subject: RE: Comments on Debian Quality Assurance (long)
Date: Fri, 12 Nov 1999 23:00:19 +0200


this is quite long, since there are so many related
issues here. I try to touch on some, but not all are even

What is quality in Debian?

When discussing quality of Debian three separate but
interconnected issues should be addressed:
A) quality of software in Debian (i.e. quality of software
   that is being packaged for Debian).
B) quality of packaging of that software (i.e. quality of
   work that is being done when adapting the software to
C) quality of Debian system (i.e. selection of suitable
   packages and defining how they should be configured to
   make an integrated whole).
Often these issues are confused, which makes discussion
difficult at times.

How these issues are handled:

A) is easiest to define, since Debian is primarily a packaging
   organization. Probably less than 5% of packages are Debian
   specific. Quality of software is dependent on upstream
   quality. Debian supports it by Bug Tracking System and by
   forwarding any bugs, fixes and enhancements upstream.

   Some maintainers do more but that is not required. Also doing
   more is conceptually difficult, since having a version that
   is modified from upstream is considered forking by many.
   Forking is considered inherently bad for free software.

B) is mainly handled by Debian Policy document and policy
   conformance checking tool Lintian. Policy is mainly discussed
   on list 'debian-policy'. The other main document is the
   Debian Developer's Reference. There are some other documents
   available in Developers Corner. There are also several tools
   to partially automate package creation and maintenance. These
   are designed to eliminate some of the most common errors.

C) is also handled by Policy and various subpolicies (e.g. Perl
   policy for Perl packages, emacsen policy, etc.)

Technical issues related to all of A) to C) are often brought
up on all mailing lists and discussed in depth, often with great

Quality assurance in voluntary organization

Modern quality thinking seems to place much emphasis on planned
quality and then verifying that planned quality has been achieved.
This is a difficult one for a large voluntary project like Debian.
Here decisions are made mostly by discussion and consensus, not as
managerial decisions. Also there is no practical way to control
the work of an individual as it is being done. By necessity all
controls are either before the work or after the work. Also the
pre-work controls are weak by nature, there is no way of preventing
anybody doing any work (in any way they want to do it), the only
real control is when incorporating that into Debian. On the other
hand, even weak controls are very effective on the long run,
since the main rewards are peer recognition, which cannot be
achieved with poor quality work, and work satisfaction, ditto.

This means that one of the most important issues is induction of
new developers. This has recently been improved, but could still
be better. There is also the list 'debian-mentors' for asking any
packaging related question or requesting private tutoring on
packaging. Other important issue is keeping maintainers active
for long time. This has been recognized but no solution has been
found yet.

There are almost no enforced reviews or strictly enforced check
points, but there is a very strong culture of peer reviews. Even
though change management and configuration control are rather
informal I still consider the achieved result to be at least as
good as in many commercial organizations. This is mainly due to
very strong commitment to quality in Debian and community effort
in doing the work.

There has been several times discussion on 'debian-test' list about
testing framework and requiring developers to supply some
regression tests or at least list of functionality to test, but
this has not yet led to any concrete requirements for developers.
There is an open policy proposal, however.

Defined processes for quality

Debian tries to be *very anti-bureaucratic*, but some processes
have been defined. Most important for quality are probably these:

- accepting and purging maintainers
   these try to ensure commitment to free software, not any
   skill set or experience level, motivation is considered
   proven by application to voluntary organization
- adopting and orphaning packages
   if there are more than one applicant then previous
   maintainer or other respected person selects; if maintainer
   cannot maintain the package he eventually orphans package
- non-maintainer uploads
   for critical bugs or packages that are not maintained
- uploading packages
   includes authentication, quality control and bug handling
- making releases and point releases of Debian distribution
   stable release, freeze, release critical issues,
   distribution testing
- bug handling
   recommended response times, proper bug closing
- resolving technical disputes
   discussion on relevant list, if no consensus then resolution by
   technical committee or by general vote

Other resources

There is no single list or web site for QA (in the modern sense)
but here are some of the most relevant ones:

Debian QA group:
 mailing list 'debian-qa'
 web sites:
Debian Test group
 list 'debian-testing' in the 'Other' section of the mailing
  lists page.
Debian Policy group
 list 'debian-policy'
 proposals are tracked in Bug Tracking System

Closing comments

Perhaps you would briefly summarize the difference in QA and QC.
There are different wievs on that, and it would be nice to know
your standpoint.

Please note, that almost all of the activities (check ...) are
actually guidelines for selecting where the improvement work
should be directed, not what I'd call pure quality control.

As you have noticed, there is no QA reference model, but if
somebody (you?) is willing to take the initiative he will
certainly receive support from many smart, motivated and
knowledgeable persons.

There is no (and probably cannot be any) design control at package
level, but there has been quite a lot of effort at distribution
level. Mostly that is codified in Policy and Developers Reference.

Even if the title Quality Assurance is not totally accurate it
probably should be preserved for historical reasons. Also
activities can always be extended in the right direction, if
volunteers are available.

 I have been hanging around and reading many
 lists since '95, but I'm not a developer, so treat this as
 an informed opinion but without any authority.


Giovanni Biscuolo <teknofin@libero.it> Wed Nov 10, 1999 3:22 PM
> I'm reading your page (http://www.debian.org/distrib/) about
> QA in Debian and was very
> excited reading the title Quality Assurance for I am an
> enthusiast of Quality culture. I think it's a great thing
> for GNU Software.
> Anyway, I would like to point out that all the activities you
> mention there (check ...) are about Quality Control, not
> Assurance. It's not my intention to teach you the difference
> - I have nothing to teach - but the difference is big.
> I'm very interested, for example, to know your QA reference
> model (ISO 9001 i.e.), how do you plan design control and,
> why not, if you are going  to develop
> and publish a GNU/Debian QA Manual.
> I think your WWW page should specify deeper the activities 
> and documents about Quality Assurance
> or you should change the title to "Quality Control".
> Last but not least: is there a mailing list about this topic?

To UNSUBSCRIBE, email to debian-qa-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

----- End forwarded message -----

Reply to: