[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: package testing, autopkgtest, and all that



On Thu, Jan 27, 2011 at 02:45:57PM +0000, Ian Jackson wrote:
> Stefano Zacchiroli writes ("package testing, autopkgtest, and all that"):
> > Regarding this specific point (tests run on packages as if they were
> > installed), IIRC Ian Jackson worked a bit on the matter, producing some
> > code (autopkgtest---as mentioned elsewhere in this thread) and a
> > specification of the interaction among tests and packages.  Ian: would
> > you mind summarizing the status of that effort and comment on whether,
> > in your opinion, people interested on this topic should continue from
> > there or start over?
> 
> Sure.  autopkgtest (the codebase) isn't very big but it contains
> several interlocking parts.  The key parts are:
> 
>  * A specification which allows a source package to declare that it
>    contains tests, and how those tests need to be run.  This
>    specification was discussed extensively on debian-devel at the
>    time and a copy is in the autopkgtest package, but I'll follow up
>    this email with a copy of it.
> 
>  * Machinery to interpret those declarations, and:
>     - build the package (if needed)
>     - install the package(s) needed for the runtime tests
>     - run the tests (if any) and collect the results
> 
>  * Some surrounding ad-hoc shell scripts and crontab code to:
>     - select a package to test
>     - run the test
>     - send the results in a fairly raw form to a webserver host
>     - make some notes about how the test went for the benefit of the
>        selection algorithm
> 
>  * A standardised interface to a virtualisation/snapshot testbed, with
>    three implementations: Xen VMs and LVM snapshots; chroot; or
>    simply running things on the actual host.
> 
> All of this seemed to work reasonably well.  The 1.2.0 in the archive
> is essentially identical to my bzr head so all the autopkgtest code is
> out there.

Excellent. I've read your followup email and the spec seems very good
(for my purposes).

> The problems are that:
> […]

As it seems to me, right now this is most useful for individual
maintainers to declare, and run, their own tests to ensure the built
packages are fine. A good start, I'd say.

> > Having a specification and some code to run the tests early on in the
> > Wheezy release cycle would be amazing, as it will enable maintainers to
> > add tests to their packages during the expected package updates for
> > Wheezy.
> 
> Absolutely.
> 
> If someone would like to set up a machine running these tests and
> perhaps do some of the qa.debian.org integration, I would be
> delighted.  

I think even without a full archive integration, having this spec
publicised a bit among developers would be useful; I know I'm looking
forward to adapting my own packages.

thanks,
iustin

Attachment: signature.asc
Description: Digital signature


Reply to: