RE: [Fuego] [PATCH] autopkgtest: add support for debian package tests
> -----Original Message-----
> From: Daniel Sangorrin on Tuesday, November 21, 2017 12:55 AM
> autopkgtest allows running tests included in Debian
> source packages. This commits adds support for running
> autopkgtest on the target (it needs to be installed)
> and parse the results.
>
> One important thing to notice is that autopkgtest installs
> packaged dependencies (testbed) and therefore leaves the
> board's Debian in a different state. For that reason,
> it is recommended that an OS deploy phase (e.g. using LAVA)
> is executed before the test.
Longterm it would be nice to have a setup and cleanup for
this test, that detected if autopkgtest needed to be installed,
and did so, and kept a record of installed packages before the
test. And then after the test removed all packages that were
installed before the test.
>
> The spec currently contains very few tests but there
> are many more available: https://ci.debian.net/data/packages/
>
> The patch and test specs were tested on BeagleBone Black
> using bone-debian-9.2-iot-armhf-2017-10-10-4gb.img.xz
I tried to test this here, but my images are too old.
See one comment below, but otherwise it's applied as is.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/tests/Functional.autopkgtest/fuego_test.sh | 19
> +++++++++++++++++
> engine/tests/Functional.autopkgtest/parser.py | 25
> +++++++++++++++++++++++
> engine/tests/Functional.autopkgtest/spec.json | 11 ++++++++++
> 3 files changed, 55 insertions(+)
> create mode 100755 engine/tests/Functional.autopkgtest/fuego_test.sh
> create mode 100755 engine/tests/Functional.autopkgtest/parser.py
> create mode 100644 engine/tests/Functional.autopkgtest/spec.json
>
> diff --git a/engine/tests/Functional.autopkgtest/fuego_test.sh
> b/engine/tests/Functional.autopkgtest/fuego_test.sh
> new file mode 100755
> index 0000000..5aac549
> --- /dev/null
> +++ b/engine/tests/Functional.autopkgtest/fuego_test.sh
> @@ -0,0 +1,19 @@
> +function test_pre_check {
> + assert_define FUNCTIONAL_AUTOPKGTEST_PACKAGES
> + is_on_target autopkgtest AUTOPKGTEST_PROGRAM /usr/bin
> + assert_define AUTOPKGTEST_PROGRAM
The "standard" (recently adopted), is to put the static portion
of the test variable name at the beginning. So I changed
this to PROGRAM_AUTOPKGTEST.
> +}
> +
> +function test_run {
> + report "echo 'Running tests for these packages:
> $FUNCTIONAL_AUTOPKGTEST_PACKAGES'"
> + for PACKAGE in $FUNCTIONAL_AUTOPKGTEST_PACKAGES; do
> + report_append "cd $BOARD_TESTDIR/fuego.$TESTDIR; mkdir
> $PACKAGE; cd $PACKAGE; \
> + apt-get source $PACKAGE; cd $PACKAGE-*; autopkgtest --output-dir
> ./output-dir-$PACKAGE -B . -- null; \
> + echo 'Fuego test_set: $PACKAGE' >>
> $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt; \
> + cat ./output-dir-$PACKAGE/summary >>
> $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt"
> + done
> +}
> +
> +function test_fetch_results {
> + get $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt
> $LOGDIR/summary.txt
> +}
> diff --git a/engine/tests/Functional.autopkgtest/parser.py
> b/engine/tests/Functional.autopkgtest/parser.py
> new file mode 100755
> index 0000000..98a2024
> --- /dev/null
> +++ b/engine/tests/Functional.autopkgtest/parser.py
> @@ -0,0 +1,25 @@
> +#!/bin/python
> +
> +import os, re, sys
> +
> +sys.path.insert(0, os.environ['FUEGO_CORE'] + '/engine/scripts/parser')
> +import common as plib
> +
> +AUTOPKGTST_SUMMARY = plib.LOGDIR + "/summary.txt"
> +
> +print "Parsing: " + AUTOPKGTST_SUMMARY
> +
> +results = {}
> +with open(AUTOPKGTST_SUMMARY) as f:
> + for line in f:
> + print line.strip()
> + if line.startswith("Fuego test_set:"):
> + test_set = line.split()[2]
> + continue
> + if line.strip():
> + print line.strip()
> + fields = line.split()
> + test_case = fields[0]
> + results[test_set+'.'+test_case] = "PASS" if fields[1] == 'PASS' else
> "FAIL"
> +
> +sys.exit(plib.process(results))
> diff --git a/engine/tests/Functional.autopkgtest/spec.json
> b/engine/tests/Functional.autopkgtest/spec.json
> new file mode 100644
> index 0000000..afa8653
> --- /dev/null
> +++ b/engine/tests/Functional.autopkgtest/spec.json
> @@ -0,0 +1,11 @@
> +{
> + "testName": "Functional.autopkgtest",
> + "specs": {
> + "smoketest": {
> + "packages": "fakesleep"
> + },
> + "default": {
> + "packages": "libzerg libproxy fakesleep"
> + }
> + }
> +}
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
Reply to: