[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Author Tests (AUTOMATED_TESTING, RELEASE_TESTING)



Hi all:

So after some discussion with Alias in the toolchain coordination
channel (the people working on Module::Install, Module::Build,
Dist::Zilla, and to a lesser extent, ExtUtils::MakeMaker), the general
consensus seems to be that using "RELEASE_TESTING=1" in general is a
harmful thing to do, and instead we should prefer to build using
"AUTOMATED_TESTING=1" only.

What this means is that Test::Pod and Test::Pod::Coverage are likely
not going to be run, whereas they are run now. It is clear that
Test::Kwalitee and Test::Perl::Critic tests, when run, could cause
some build failures down the line, so our standard is already not to
run these because they might fail. By extension, Adam Kennedy contends
that Test::Pod and Test::Pod::Coverage are similar. Though these
modules don't change often, if they do, it has the risk of failing
where things were previously passing, resulting in FTBFS errors.

22:29:03 < jawnsy> Alias: I guess, then, the question becomes: should
Debian run Release Tests
22:29:10 <@Alias> No
22:29:12 <@mst> no
22:29:20 <@Alias> That's the entire reason I lobbied for the rename
from "author" to "release"
22:29:24  * xdg doesn't care enough

Thoughts from everyone else?

I know gregor prefers us to run as many tests as possible; however, in
light of their arguments, I now think it's safer for us (not to
mention less maintenance work) to just run AUTOMATED tests, and no
longer run RELEASE tests, except in rare occasions. One case is where
the upstream maintainer of DBIx::Class has told us that running
RELEASE_TESTING tests would be particularly useful for detecting
certain regressions.

I propose we drop the "run RELEASE_TESTING tests by default" and
consider each of these modules carefully.

Cheers,

Jonathan


Reply to: