Re: "Debian CI/autopkgtest BoF" at Debconf18

[Ian Jackson <ijackson@chiark.greenend.org.uk>]

gobby notes from the bof:

Ian.


Debian CI/autopkgtest BoF
=========================

## unsatisfiable test-dependencies should not be a test failure
DON'T use apt in your script
Use flaky restriction + it needs a fix in autopkgtest to properly support that
  (as mentioned by Ralf Treinen, it doesn't work yet).
bug to be filed


## trivial tests
trivial tests: no faster migration, but with delays (or block in the future) on 
failure.
One can implement now with skippable restriction and exit 77 on success
See https://bugs.debian.org/904979


## Best practices
https://wiki.debian.org/ContinuousIntegration/AutopkgtestBestPractices
please add yours


## autodep8 and smarter
To *add* tests to autodep8 tests, set "Testsuite: autopkgtest-pkg-<class>" in
d/control and put the tests in d/tests/control.autodep8. But please be aware,
_maybe_ control.autodep8 will be deprecated, please check bug 
https://bugs.debian.org/904776


## Running autopkgtest on every commit/push to Salsa

https://salsa.debian.org/salsa-ci-team/pipeline/blob/master/salsa-ci.yml#L28-32

Otto testing out at https://salsa.debian.org/mariadb-team/mariadb-10.1/-/jobs - 
will write about experience/findings later.
Note: In the Debian context, remember to put your .gitlab-ci.yml files in debian/.gitlab-ci.yml and configure the new location at https://salsa.debian.org/<project>/<repo>/settings/ci_cd


## figuring out what changed when tests fail

autopkgtest does list which packages (w/ versions) were installed for the tests.
On ci.debian.net this is always in the artifacts tarball, on unstable there is
also the log link, but beware https://bugs.debian.org/812856
(the artifacts is always right AFAIK)


## Adding indirect reverse depends to your tests:
Use a dummy test, add Restriction: hint-testsuite-triggers and add your triggers
to the Depends
Ian should still create his Merge Request ;) discussion in the thread starting
https://lists.debian.org/msgid-search/23335.38346.638278.79737@chiark.greenend.org.uk


## test doesn't always blame the right package
Maybe britney should test recursive dependencies as well?




-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.