Automatic testing of .deb's
(Note crosspost to debian-devel and ubuntu-devel. If your reply does
not concern both Debian and Ubuntu, please remove the inappropriate
list. And of course, in general, do not CC individual posters unless
they ask for it.)
Following discussions at Ubuntu's Montreal meeting, on debian-devel,
and elsewhere, I've done an initial implementation of my planned
automatic testing framework. In many ways it is still a proof of
concept, and it lacks many useful features, but it can already be used
to test packages in a chroot in a basic way.
The package with the test machinery is `autodebtest'[1] and can be
found[2] in Ubuntu dapper and Debian sid. The dependencies are
relatively conservative so I think the .deb should install and work
with breezy or sarge too.
There is currently only one test in one package. I picked `mawk' as
an example because it is a package which is (i) very stable (ii) has
limited dependencies and (iii) has at least some upstream test suite.
The new test runs the provided upstream mawk tests (they are not
separated out by the upstream test suite so it looks like one test
from the autodebtest point of view).
This new version of the mawk package has not been published anywhere
yet. So, I have appended the (small) patch to this message.
Next on my todo list for this project is to try to integrate piuparts
somehow. This will mean inventing some machinery for `any package'
tests.
Should I file the mawk patch, attached, as a wishlist bug against
mawk[3] ? It would be nice to be able to integrate the tests for each
package as far upstream as possible; at the moment, that probably
means putting it in the Debian packages.
I would like to have some idea what people think I should do with the
tests that we're hopefully going to have, eventually for lots of
packages. Would Debian like those tests as patches in wishlist bug
reports, in general ? That would seem to be best to me but before I
go down this route I'd like to be clear that that's what Debian
developers want.
Below you can find a demo transcript.
Thanks,
Ian.
[1] The package will be renamed `autopkgtest' in my next upload.
We apologise for the inconvenience.
[2] You need 0.5.2 which is currently awaiting archive processing for
both sid and dapper. Currently autodebtest is in dapper universe; the
renamed package will be in main RSN. You can find a copy here:
[3] I have BCC'd the Debian mawk maintainer.
http://www.chiark.greenend.org.uk/~ian/autodebtest-tmp/
liberator:~> dchroot
Executing shell in 'dapper' chroot.
To execute a command as root, run "sudo <command>" and enter your password.
ian@liberator:~$ cd /work/AutomatedTesting/demo
ian@liberator:/work/AutomatedTesting/demo$ mkdir unpatched
ian@liberator:/work/AutomatedTesting/demo$ cd unpatched/
ian@liberator:/work/AutomatedTesting/demo/unpatched$ dpkg-source -x /work/download/mawk_1.3.3-11.dsc
dpkg-source: extracting mawk in mawk-1.3.3
dpkg-source: unpacking mawk_1.3.3.orig.tar.gz
dpkg-source: applying /work/download/mawk_1.3.3-11.diff.gz
ian@liberator:/work/AutomatedTesting/demo/unpatched$ cd mawk-1.3.3/
ian@liberator:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ debian/rules build >../build-log 2>&1
ian@liberator:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ echo $?
0
ian@liberator:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ fakeroot debian/rules binary >../binary-log 2>&1
ian@liberator:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ echo $?
0
ian@liberator:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ cd ../..
ian@liberator:/work/AutomatedTesting/demo$ mkdir patched
ian@liberator:/work/AutomatedTesting/demo$ cd patched/
ian@liberator:/work/AutomatedTesting/demo/patched$ dpkg-source -x /work/download/mawk_1.3.3-11.dsc
dpkg-source: extracting mawk in mawk-1.3.3
dpkg-source: unpacking mawk_1.3.3.orig.tar.gz
dpkg-source: applying /work/download/mawk_1.3.3-11.diff.gz
ian@liberator:/work/AutomatedTesting/demo/patched$ cd mawk-1.3.3/
ian@liberator:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ patch -p1 -s </work/AutomatedTesting/mawk-1.3.3.adt.diff
ian@liberator:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ debian/rules build >../build-log 2>&1
ian@liberator:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ echo $?
0
ian@liberator:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ fakeroot debian/rules binary >../binary-log 2>&1
ian@liberator:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ echo $?
0
ian@liberator:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ logout
liberator:~> really dchroot
Executing shell in 'dapper' chroot.
root@liberator:~# cd /work/AutomatedTesting/demo/unpatched/
root@liberator:/work/AutomatedTesting/demo/unpatched# dpkg -i mawk_1.3.3-11_i386.deb
(Reading database ... 70650 files and directories currently installed.)
Preparing to replace mawk 1.3.3-11 (using mawk_1.3.3-11_i386.deb) ...
Unpacking replacement mawk ...
Setting up mawk (1.3.3-11) ...
root@liberator:/work/AutomatedTesting/demo/unpatched# logout
liberator:~> adt-run --output-dir ~/d --build-tree-tb /work/AutomatedTesting/demo/patched/mawk-1.3.3 --- adt-virt-chroot =dapper
mawktest PASS
liberator:~> cat d/stderr-mawktest
liberator:~> cat d/stdout-mawktest
mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan
testing input and field splitting
input and field splitting OK
testing regular expression matching
regular expression matching OK
testing checking for write errors
checking for write errors OK
testing arrays and flow of control
array test OK
testing function calls and general stress test
general stress test passed
tested mawk seems OK
liberator:~> really dchroot su -
(dapper) su -
root@liberator:~# dpkg --purge --force-depends mawk
dpkg: mawk: dependency problems, but removing anyway as you request:
ubuntu-minimal depends on mawk.
base-files depends on awk; however:
Package awk is not installed.
Package mawk which provides awk is to be removed.
(Reading database ... 70649 files and directories currently installed.)
Removing mawk ...
root@liberator:~# logout
liberator:~> adt-run --output-dir ~/d --build-tree-tb /work/AutomatedTesting/demo/patched/mawk-1.3.3 --- adt-virt-chroot =dapper
mawktest FAIL stderr: /tmp/tmp.9omUyv/mawktest: line 24: mawk:...
liberator:~> cat d/stderr-mawktest
/tmp/tmp.9omUyv/mawktest: line 24: mawk: command not found
liberator:~>
diff --exclude 'parse.[ch]' -ruN orig/mawk-1.3.3/debian/tests/control mawk-1.3.3/debian/tests/control
--- orig/mawk-1.3.3/debian/tests/control 1970-01-01 01:00:00.000000000 +0100
+++ mawk-1.3.3/debian/tests/control 2006-01-31 15:25:36.081499137 +0000
@@ -0,0 +1,2 @@
+Tests: mawktest
+Restrictions: rw-build-tree
diff --exclude 'parse.[ch]' -ruN orig/mawk-1.3.3/debian/tests/mawktest mawk-1.3.3/debian/tests/mawktest
--- orig/mawk-1.3.3/debian/tests/mawktest 1970-01-01 01:00:00.000000000 +0100
+++ mawk-1.3.3/debian/tests/mawktest 2006-01-31 15:25:36.081499137 +0000
@@ -0,0 +1,9 @@
+#!/bin/sh
+set -e
+sed -e '
+ /^PATH=/d;
+ /^mawk -W version/ s/$/ 2>&1/;
+' <test/mawktest >"$TMPDIR/mawktest"
+chmod +x "$TMPDIR/mawktest"
+cd test
+"$TMPDIR/mawktest"
--
Reply to: