On Thu, Jan 27, 2011 at 02:45:57PM +0000, Ian Jackson wrote:
> Stefano Zacchiroli writes ("package testing, autopkgtest, and all that"):
> > Regarding this specific point (tests run on packages as if they were
> > installed), IIRC Ian Jackson worked a bit on the matter, producing some
> > code (autopkgtest---as mentioned elsewhere in this thread) and a
> > specification of the interaction among tests and packages. Ian: would
> > you mind summarizing the status of that effort and comment on whether,
> > in your opinion, people interested on this topic should continue from
> > there or start over?
>
> Sure. autopkgtest (the codebase) isn't very big but it contains
> several interlocking parts. The key parts are:
>
> * A specification which allows a source package to declare that it
> contains tests, and how those tests need to be run. This
> specification was discussed extensively on debian-devel at the
> time and a copy is in the autopkgtest package, but I'll follow up
> this email with a copy of it.
>
> * Machinery to interpret those declarations, and:
> - build the package (if needed)
> - install the package(s) needed for the runtime tests
> - run the tests (if any) and collect the results
>
> * Some surrounding ad-hoc shell scripts and crontab code to:
> - select a package to test
> - run the test
> - send the results in a fairly raw form to a webserver host
> - make some notes about how the test went for the benefit of the
> selection algorithm
>
> * A standardised interface to a virtualisation/snapshot testbed, with
> three implementations: Xen VMs and LVM snapshots; chroot; or
> simply running things on the actual host.
>
> All of this seemed to work reasonably well. The 1.2.0 in the archive
> is essentially identical to my bzr head so all the autopkgtest code is
> out there.
Excellent. I've read your followup email and the spec seems very good
(for my purposes).
> The problems are that:
> […]
As it seems to me, right now this is most useful for individual
maintainers to declare, and run, their own tests to ensure the built
packages are fine. A good start, I'd say.
> > Having a specification and some code to run the tests early on in the
> > Wheezy release cycle would be amazing, as it will enable maintainers to
> > add tests to their packages during the expected package updates for
> > Wheezy.
>
> Absolutely.
>
> If someone would like to set up a machine running these tests and
> perhaps do some of the qa.debian.org integration, I would be
> delighted.
I think even without a full archive integration, having this spec
publicised a bit among developers would be useful; I know I'm looking
forward to adapting my own packages.
thanks,
iustin
Attachment:
signature.asc
Description: Digital signature