[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

mass autopkgtest Perl run [2014-09-20]

On Sat, Sep 20, 2014 at 02:15:13PM +0200, Martin Pitt wrote:
> Niko Tyni [2014-09-18 23:56 +0300]:
> > Once #762013 (autopkgtest: detect test.pl in perl packages) is fixed, I
> > think one more full rebuild with that and pkg-perl-autopkgtest 0.16 would
> > be good as running test.pl will probably pick up a few new failures too.
> I ran the whole batch again yesterday. As usual the logs are on
>   https://people.debian.org/~mpitt/tmp/logs-20140920.tar.xz
>   https://people.debian.org/~mpitt/tmp/perl-failed-tests.txt
>   https://people.debian.org/~mpitt/tmp/perl-failures/

Thanks once again.

> With the new pkg-perl-autopkgtest 0.16 

The logs look like you only had pkg-perl-autopkgtest 0.15 available.
The main change is that t/boilerplate.t tests are ignored with 0.16.
My tests indicate this made 31 new packages pass.

> there are 37 new successes, but
> also 32 regressions, i. e. these passed earlier and now fail:

> I think these new failures deserve a closer look. 

Right. I went through these, and found indeed a couple of bugs in
pkg-perl-autopkgtest. I've just uploaded 0.17 which fixes 21 of those 32
regressions. The rest are either real bugs in the packages or tradeoffs:

- including dummy MANIFEST+MANIFEST.SKIP files in the test directory
  makes more new passing packages than new failed ones (I think).

  to do even though it activates author tests needing extra dependencies
  in some buggy packages

>   https://people.debian.org/~mpitt/tmp/perl-failures/libapp-termcast-perl/auto-build-deps-stdout

FWIW this particular one is due to a new upstream release that now needs
files outside t/, so it requires a bit of manual tuning.

I see libgnome2-vfs-perl used to skip tests with "You have no ~/.gnome"
but didn't in the last go, so presumably your test setup has changed a bit.
(The package is probably buggy if it looks in $HOME, though; bug filed.)

libextutils-autoinstall-perl got killed due to a timeout, but builds
fine for me. Is that reproducible for you?

> > Unless I've broken something, I think the passing packages from that
> > could go live on the ci.debian.net whitelist and we'll start fixing the
> > remaining ones and adding the explicit Testsuite headers.
> Yes, I agree. So far I kept perl.txt up to date with new
> pkg-perl-autopkgtest versions, so at no point there should be any new
> failures. (I'm sure there will be a handful due to flaky tests, timing
> issues, etc.)

Yeah, at least libtest-cgi-multipart-perl is undeterministic (and RC-buggy
because of that.)

So would you like me to update the whitelist with the 32+21 new passing
packages, or do we do one more test run with pkg-perl-autopkgtest 0.17?
(In the latter case, waiting for the binNMUs in #760123 would be prudent
as we're currently in a small Perl transition.)

PS: I've just documented all this a bit at
Comments welcome.
Niko Tyni   ntyni at debian.org

Reply to: