Re: mass autopkgtest Perl run [2014-09-20]
On Sat, Sep 20, 2014 at 02:15:13PM +0200, Martin Pitt wrote:
> Niko Tyni [2014-09-18 23:56 +0300]:
> > Once #762013 (autopkgtest: detect test.pl in perl packages) is fixed, I
> > think one more full rebuild with that and pkg-perl-autopkgtest 0.16 would
> > be good as running test.pl will probably pick up a few new failures too.
> I ran the whole batch again yesterday. As usual the logs are on
Thanks once again.
> With the new pkg-perl-autopkgtest 0.16
The logs look like you only had pkg-perl-autopkgtest 0.15 available.
The main change is that t/boilerplate.t tests are ignored with 0.16.
My tests indicate this made 31 new packages pass.
> there are 37 new successes, but
> also 32 regressions, i. e. these passed earlier and now fail:
> I think these new failures deserve a closer look.
Right. I went through these, and found indeed a couple of bugs in
pkg-perl-autopkgtest. I've just uploaded 0.17 which fixes 21 of those 32
regressions. The rest are either real bugs in the packages or tradeoffs:
- including dummy MANIFEST+MANIFEST.SKIP files in the test directory
makes more new passing packages than new failed ones (I think).
- setting AUTOMATED_TESTING and NONINTERACTIVE_TESTING is the right thing
to do even though it activates author tests needing extra dependencies
in some buggy packages
FWIW this particular one is due to a new upstream release that now needs
files outside t/, so it requires a bit of manual tuning.
I see libgnome2-vfs-perl used to skip tests with "You have no ~/.gnome"
but didn't in the last go, so presumably your test setup has changed a bit.
(The package is probably buggy if it looks in $HOME, though; bug filed.)
libextutils-autoinstall-perl got killed due to a timeout, but builds
fine for me. Is that reproducible for you?
> > Unless I've broken something, I think the passing packages from that
> > could go live on the ci.debian.net whitelist and we'll start fixing the
> > remaining ones and adding the explicit Testsuite headers.
> Yes, I agree. So far I kept perl.txt up to date with new
> pkg-perl-autopkgtest versions, so at no point there should be any new
> failures. (I'm sure there will be a handful due to flaky tests, timing
> issues, etc.)
Yeah, at least libtest-cgi-multipart-perl is undeterministic (and RC-buggy
because of that.)
So would you like me to update the whitelist with the 32+21 new passing
packages, or do we do one more test run with pkg-perl-autopkgtest 0.17?
(In the latter case, waiting for the binNMUs in #760123 would be prudent
as we're currently in a small Perl transition.)
PS: I've just documented all this a bit at
Niko Tyni email@example.com