Re: package testing, autopkgtest, and all that

[Lars Wirzenius <liw@liw.fi>]

On ke, 2011-02-02 at 17:13 +0000, Ian Jackson wrote:
> This is easily done with autopkgtest; the only difference from your
> proposal is that the source package needs to be downloaded.  Doing so
> is not difficult or troublesome, and can be done automatically.

I concur.

However, looking things from a slightly different angle, I can see
several use cases for automated testing in Debian:

      * the package maintainers want to know that their package works
      * upstream wants to know we haven't broken anything in Debian
      * release managers (and everyone else!) want to know the disto in
        general is ready to release, and also that any one updated
        package doesn't break other stuff (e.g., libpng might break web
        browsers using it)
      * security team wants to know the security update doesn't break
        anything
      * sysadmins want to know upgrading to a new release doesn't break
        their servers
      * I want to know I can upgrade to today's version of testing
        without anything major breaking

Obviously, no amount of testing will be able to find all breakage, but
even a relatively low test coverage can be extremely helpful, if it is
targeted suitably.

autopkgtest seems to me to target package maintainers, release managers,
and the security team, from the list above.

A different toolset and possibly testset might be necessary for other
use cases. Once we have those tools, we can use them, but for now
concentrating on autopkgtest is probably the best idea.

-- 
Blog/wiki/website hosting with ikiwiki (free for free software):
http://www.branchable.com/