On Thu, Feb 01, 2001 at 11:50:35PM +0100, Marcelo E. Magallon wrote:
> Once we have the people, the other part of the answer is not that
> clear. IMO, a possible solution would be to have a sort of test suite.
> Since the size of Debian together with the size of the QA team makes
> the very idea of a generalized test suite laughable at the moment.
> Phil did develop such a beast, but if you look in /usr/lib/debian-test,
> unless you have a particular set of packages, you'll note it's empty.
> Woody's release is hopefully too close to attempt getting these scripts
> written for a significant part of Debian.[1]
There are actually at least three sorts of tests we can do:
* unit / code-based tests, as part of the general debian/rules
process: these are really good in that they'll automagically be
run on each architecture the package gets built for, and if they
fail the package won't build and people might even notice.
* .deb verification stuff: lintian is an obvious example. We can
also do things like check that if any two .debs in the same
arch/suite have the same file, then they Conflict:. We can also
check packages don't depend on lower priority packages, and only
conflict with things in extra. All automatically.
* real environment stuff, which is what debian-test does. If you've
installed an mp3 player, and you've got sound hardware, you can
run some stuff and say "yes, that sound came from my left speaker
and that other one came from my right speaker"
There's probably some helpful infrastructure that could be made available
for all these (for the first, a debhelper-like package could be useful,
eg). In any event, it'd probably be helpful if some people tried writing
some tests and sent patches to maintainers, or published weekly reports
or whatever's appropriate.
> For now, it seems the most urgent task in this yet non-existant list is
> to test the boot-floppies. For *woody*.
Which is a nice idea in principle, but it's hard to test something that
doesn't really exist...
> The other question that popped up is "what's holding up packages in
> unstable and what can QA do to help that?"
Go through the update_excuses list, find a package that looks interesting
and get rid of any excuses that're holding it up, look at autobuilder logs
if they're available, send in patches to bugs, etc. Make NMUs if necessary
as usual. If there aren't any excuses holding it up (it's a "valid candidate")
have a look at the update_output list, for a line like:
ace: alpha: ivtools-bin ivtools-dev ivtools-unidraw
which means the "ace" source package (along with whatever binary packages
are compiled for it) wasn't added to woody (from unstable) because doing
so would've made ivtools-{bin,dev,unidraw} uninstallable on alpha.
Only the first (alphabetically) archicture where things break is listed
for efficiency purposes. :-/
Following these through is a bit arduous, but tends to end up showing
you a bug that can be fixed, so it's not usually a waste of time.
I'm treating debian-testing as a group that finds problems and can judge
how functional or buggy the distribution is, and debian-qa as a group
that fixes problems and has tools and procedures to try to stop them
reoccuring. Both these things really come under the heading "quality
assurance", but we've got two different lists, so we may's'well use 'em.
FWIW, YMMV, etc.
Cheers,
aj (woody release manager, like you didn't already know)
--
Anthony Towns <aj@humbug.org.au> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. GPG signed mail preferred.
``_Any_ increase in interface difficulty, in exchange for a benefit you
do not understand, cannot perceive, or don't care about, is too much.''
-- John S. Novak, III (The Humblest Man on the Net)
Attachment:
pgpjZtAt5AaB5.pgp
Description: PGP signature