Hello again, I'm starting a new thread to keep the original one for the generic discussion and this to fixing the particular failures. Yesterday I started another run with the complete batch and let it run over night. To avoid the problem of having to kill hanging tests (most often just in their cleanup, the actual tests usually work fine) I ran them with --timeout-test=300 (i. e. 5 minutes). This killed 14 tests (grep for "testbed failed: timed out"). Apologies if this killed a legitimately running test, I'm happy to re-run those without a short timeout (or you just do it yourself). FYI, the whole run took 11 hours; that's with a filled apt-cacher-ng, directory (i. e. not tarball) schroot, and putting overlays in tmpfs. The exit code bug indeed caused a lot of false passes, now the numbers look quite a bit worse: Total: 2713 <https://people.debian.org/~mpitt/tmp/perl-all.txt> Passing: 1765 (65%) Failing: 948 (35%) <https://people.debian.org/~mpitt/tmp/perl-failed-tests.txt> As before, the failed logs are in https://people.debian.org/~mpitt/tmp/perl-failures/ This time I kept the successful logs as well; for easier grepping etc. I prefixed the directories with "fail-" and "pass-": https://people.debian.org/~mpitt/tmp/logs-20140911.tar.xz (8 MB compressed, 367 MB uncompressed) The command to run this: for p in $(< pkgs); do if env -u DISPLAY adt-run -o $p --timeout-test=300 $p --- schroot sid; then mv $p pass-$p; else mv $p fail-$p; fi; done (autopkgtest 3.5) I updated debian-ci-config with the new failures. Martin -- Martin Pitt | http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
Attachment:
signature.asc
Description: Digital signature