Bug#282311: defining the qa process
Package: qa.debian.org
Version: unavailable; reported 2004-11-20
Severity: Important
I just took a closer look at qa.debian.org and discovered it is more
of a generalized (ie not one package) bug database than a place to
develop debian qualifications, which is the heart of qa. It would
seem there are no qualification guidelines from which to benchmark
quality, only policy which generalizes requirements but says very
little about process, and nothing about why such process is correct.
Defining what a package.postinst script supports, with detailed
requirements, resulting specification, and complementing it with a
script to validate its execution did what was intended would go a
very long way toward qa.
Presently, I think there is a general assumed supported and
requirements. Something like, "you must have installed from cdroms
and dist-upgraded before your version became unsupported" but this
is far to general. For one it doesn't define what you can and cannot
change (eg interfaces, resolv.conf, but not /etc/init.d/*). If the
qualifications specify only the "debian (cdrom) way" they will be
totally useless for anything but the warm and fuzzy factor. The
real value for QA is the hooks it provides for ISO-9002 or other
regulated installations. In more than 9 out of 10 installations
_something_ must be done that deviates from the debian way; but if
_anything_ deviates from the spec that defines what is supported,
the qualification is invalidated and useless.
Therefore, a package spec and requirements should outline the very
minimum and generic needed to satisfy its qualification. ie to
install apache, QA should define 1) port 80 must be available; 2)
files in apache.list will be overwritten; 3) apache.postinst expects
a, b and c, and does x, y and z; 3) etc.
Please see additional discussion at
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=282306
Not only did awk fail but apt-get didn't notice. I do log access
outside of /var/log/apache and I do pipe errors to an external program
for processing. I don't think any of this is terribly unusual.
and
http://leaf.dragonflybsd.org/mailarchive/kernel/2004-11/msg00235.html
For deploying in regulated industry, there is something else relevant
than patching security issues: qualification. Setting up regulated
systems requires defining objectives and documenting how they are met.
// George
--
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org
Reply to: