[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Mass bugs filing: autopkgtest should be marked superficial (new list)



Hi All,

If the test done in the autopkgtest does not provide significant test
coverage then it should be marked with "Restrictions: superficial".
Ref: https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst

Examples of tests which are not significant includes (its not a complete list):

1) Executing the binary to check version
    Test-Command: foo -v
    Test-Command: foo -V
    Test-Command: foo --version

2) Executing the binary to check help (foo -h)
    Test-Command: foo -h
    Test-Command: foo --help

3) checking for files installed with 'ls'.
    Test-Command: ls -l /usr/lib/*/foo.so

4) A Python or Perl library runs import foo or require Foo; but does
not attempt to use the library beyond that.
     Test-Command: python3 -c "import foo"

I intend to file them with "severity: serious" as packages with
non-trivial autopkgtests enjoy a reduced age for migration from
unstable to testing and they are also allowed to migrate in a later
stage of the freeze than other packages. Ref:
https://release.debian.org/bullseye/freeze_policy.html

And, rc_policy confirms "These tests must test at least one of its own
installed binary packages in some way, or must be marked as
superficial". Ref: https://release.debian.org/bullseye/rc_policy.txt

Also, Paul has confirmed that if its fixed in git but not uploaded
then the severity can be reduced while marking the bug as pending.
Ref: https://lists.debian.org/debian-devel/2020/09/msg00236.html

Attached is the dd-list.


-- 
Regards
Sudip

Attachment: dd-list
Description: Binary data


Reply to: