[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

[Fuego] [PATCH] autopkgtest: add support for debian package tests



> -----Original Message-----
> From: Daniel Sangorrin on Tuesday, November 21, 2017 12:55 AM
> autopkgtest allows running tests included in Debian
> source packages. This commits adds support for running
> autopkgtest on the target (it needs to be installed)
> and parse the results.
> 
> One important thing to notice is that autopkgtest installs
> packaged dependencies (testbed) and therefore leaves the
> board's Debian in a different state. For that reason,
> it is recommended that an OS deploy phase (e.g. using LAVA)
> is executed before the test.

Longterm it would be nice to have a setup and cleanup for
this test, that detected if autopkgtest needed to be installed,
and did so, and kept a record of installed packages before the
test.  And then after the test removed all packages that were
installed before the test.

> 
> The spec currently contains very few tests but there
> are many more available: https://ci.debian.net/data/packages/
> 
> The patch and test specs were tested on BeagleBone Black
> using bone-debian-9.2-iot-armhf-2017-10-10-4gb.img.xz

I tried to test this here, but my images are too old.

See one comment below, but otherwise it's applied as is.

> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
> ---
>  engine/tests/Functional.autopkgtest/fuego_test.sh | 19
> +++++++++++++++++
>  engine/tests/Functional.autopkgtest/parser.py     | 25
> +++++++++++++++++++++++
>  engine/tests/Functional.autopkgtest/spec.json     | 11 ++++++++++
>  3 files changed, 55 insertions(+)
>  create mode 100755 engine/tests/Functional.autopkgtest/fuego_test.sh
>  create mode 100755 engine/tests/Functional.autopkgtest/parser.py
>  create mode 100644 engine/tests/Functional.autopkgtest/spec.json
> 
> diff --git a/engine/tests/Functional.autopkgtest/fuego_test.sh
> b/engine/tests/Functional.autopkgtest/fuego_test.sh
> new file mode 100755
> index 0000000..5aac549
> --- /dev/null
> +++ b/engine/tests/Functional.autopkgtest/fuego_test.sh
> @@ -0,0 +1,19 @@
> +function test_pre_check {
> +    assert_define FUNCTIONAL_AUTOPKGTEST_PACKAGES
> +    is_on_target autopkgtest AUTOPKGTEST_PROGRAM /usr/bin
> +    assert_define AUTOPKGTEST_PROGRAM

The "standard" (recently adopted), is to put the static portion
of the test variable name at the beginning.  So I changed
this to PROGRAM_AUTOPKGTEST.

> +}
> +
> +function test_run {
> +    report "echo 'Running tests for these packages:
> $FUNCTIONAL_AUTOPKGTEST_PACKAGES'"
> +    for PACKAGE in $FUNCTIONAL_AUTOPKGTEST_PACKAGES; do
> +        report_append "cd $BOARD_TESTDIR/fuego.$TESTDIR; mkdir
> $PACKAGE; cd $PACKAGE; \
> +            apt-get source $PACKAGE; cd $PACKAGE-*; autopkgtest --output-dir
> ./output-dir-$PACKAGE -B  . -- null; \
> +            echo 'Fuego test_set: $PACKAGE' >>
> $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt; \
> +            cat ./output-dir-$PACKAGE/summary >>
> $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt"
> +    done
> +}
> +
> +function test_fetch_results {
> +    get $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt
> $LOGDIR/summary.txt
> +}
> diff --git a/engine/tests/Functional.autopkgtest/parser.py
> b/engine/tests/Functional.autopkgtest/parser.py
> new file mode 100755
> index 0000000..98a2024
> --- /dev/null
> +++ b/engine/tests/Functional.autopkgtest/parser.py
> @@ -0,0 +1,25 @@
> +#!/bin/python
> +
> +import os, re, sys
> +
> +sys.path.insert(0, os.environ['FUEGO_CORE'] + '/engine/scripts/parser')
> +import common as plib
> +
> +AUTOPKGTST_SUMMARY = plib.LOGDIR + "/summary.txt"
> +
> +print "Parsing: " + AUTOPKGTST_SUMMARY
> +
> +results = {}
> +with open(AUTOPKGTST_SUMMARY) as f:
> +    for line in f:
> +        print line.strip()
> +        if line.startswith("Fuego test_set:"):
> +            test_set = line.split()[2]
> +            continue
> +        if line.strip():
> +            print line.strip()
> +            fields = line.split()
> +            test_case = fields[0]
> +            results[test_set+'.'+test_case] = "PASS" if fields[1] == 'PASS' else
> "FAIL"
> +
> +sys.exit(plib.process(results))
> diff --git a/engine/tests/Functional.autopkgtest/spec.json
> b/engine/tests/Functional.autopkgtest/spec.json
> new file mode 100644
> index 0000000..afa8653
> --- /dev/null
> +++ b/engine/tests/Functional.autopkgtest/spec.json
> @@ -0,0 +1,11 @@
> +{
> +    "testName": "Functional.autopkgtest",
> +    "specs": {
> +        "smoketest": {
> +            "packages": "fakesleep"
> +        },
> +        "default": {
> +            "packages": "libzerg libproxy fakesleep"
> +        }
> +    }
> +}
> --
> 2.7.4
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego



Reply to: