Re: Explaining Debian's approach to QA
On Mon, May 21, 2018 at 3:26 AM, Jose Miguel Parrella Romero wrote:
> Over the last few months, I've found myself struggling to find a simple
> way to describe our approach to QA to friends and colleagues. I reached
> out to lamby a week or so ago and he suggested I brought it to -devel.
The best description is that our QA is defined by the people who work
on it and distributed amongst those people according to their
interests, motivations and available time.
This includes both people involved in Debian and external people.
The wiki page provides a good overview of all the initiatives (some dead):
https://wiki.debian.org/qa.debian.org
Examples of the latter include Mayhem, repology and cppcheck:
https://forallsecure.com/blog/
https://repology.org/repository/debian_unstable/problems
http://cppcheck.sourceforge.net/devinfo/daca2-report/daca2.html
There are some gaps in the wiki page too, for example helmut does a
lot of cross-build QA.
There are some gaps in our QA too. For example we lack an ABI tracking
service using pkg-abidiff and abipkgdiff.
> (1) QA at Debian is a multi-stakeholder process* where ftp-master, QA,
> Release and other teams play a role but core accountability lies with
> the package maintainer,
This misses a few other actors; users, bug reporters, porters, NMUers,
QA uploaders, stable uploaders, external researchers etc.
> (2) The preferred vehicle for QA accountability in Debian is a bug
> report, and,
I don't think that is full picture. Many issues are only conveyed via
lintian and other automated QA tools. Also sponsor or ftp-master
reviews pick up lots of issues.
> (3) The QA "bar" is codified in Debian Policy but also in many other
> places (maybe not comprehensively) including for example the Release
> checklist
Indeed, there is no single document or tool that provides the full
picture for a package or for Debian as a whole. This is one of the
reasons I started the wiki page that lead to jwilk writing
check-all-the-things. That said, tracker.debian.org aims to aggregate
all available per-package data sources.
--
bye,
pabs
https://wiki.debian.org/PaulWise
Reply to: