Re: Not running tests because tests miss source code is not useful
* Pirate Praveen <praveen@onenetbeyond.org> [211007 05:41]:
> On 7 October 2021 3:02:55 am IST, Thomas Goirand <zigo@debian.org> wrote:
> >On 10/6/21 6:53 PM, Pirate Praveen wrote:
> >> On ബു, ഒക്ടോ 6 2021 at 12:16:07 വൈകു +0200 +0200, Jonas Smedegaard > <jonas@jones.dk> wrote:
> >>> Quoting Yadd (2021-10-06 11:43:40)
> >>>> On Lu, 04 oct 21, 16:40:48, Bastien Roucari�s wrote:
> >>>> > Source: src:node-lodash
> >>>> > Version: 4.17.21+dfsg+~cs8.31.173-1
> >>>> > Severity: serious
> >>>> > Justification: do not compile from source
> >>>> >
> >>>> > Dear Maintainer,
> >>>> >
> >>>> > The vendor directory should be emptied
> >>>> >
> >>>> > The debug version is compiled without source (lintian warn)
> >>>> > and moreover the rest of file are already packaged
> >>>> >
> >>>> > grep -R vendor * gives only a few hit that could be cured by
> >>>> > symlinking
> >>>> >
> >>>> > Bastien
> >>>> Hi,
> >>>>
> >>>> this files are used for test only, maybe severity could be decreased.
> >>>
> >>> I find the severity accurate: Relying on non-source code is a severe
> >>> violation of Debian Policy, not matter the purpose of relying on it.
> >>
> >> I think we should change the policy here. Running tests helps improve
> >> the quality of the software we ship. Many times the vendored code is
> >> used to ensure the code does not break in a specific situation. I don't
> >> think reducing test coverage in such situations is really helpful.
> >
> >Right, running tests helps improve the quality of software we ship.
> >Which is why you probably need to test using what's shipped in Debian
> >rather than using a vendored source-less code.
>
> We are not shipping the source less code. This is used only during
> tests. I don't think we are not gaining anything by removing tests
> here. Just making it harder for the package maintainer to run tests.
If the original report is correct (I haven't looked at the package),
then Debian is, indeed, shipping the source-less code. Debian ships
both source packages and binary packages. The source-less code is in
the source package, and is thus shipped by Debian. The DFSG applies to
both source and binary packages.
> >If we rely on non-free code for tests, that's really bad too, and that
> >must be avoided just like we're avoiding source-less code everywhere
> >else in Debian. The policy shall not change, please.
+1
> The code is not non-free here, just a specific version of a Free
> Software code built outside Debian.
I think you are confusing "Free Software" with "DFSG Free Software".
Debian has more strict standards than some other parts of the Free
Software community.
> I think tools required for tests should be considered separately from
> tools required to compile. I think it should be treated similar to
> test data.
Actually, I agree with this, but not exactly in the way you intended. I
think there should be a way for a source package to specifically
identify a Build-Dep as a test (probably with a separate field, maybe
Test-Depends), and to identify tests that must run for the build to be
considered successful, and tests that are optional (perhaps
Test-Suggests). This would allow Non-Free tests (and test data) to be
packaged separately and included in non-free where they belong, with the
requirement that a source package can Test-Suggests a non-free package,
but it cannot Test-Depends such a package. This would also allow tests
that require external networking resources, or tests that consumed an
inordinate amount of CPU, or..., to be Test-Suggested, without
interfering with a normal binary build.
The buildds could be enhanced to build the binary packages without any
of the test dependencies present, then install the desired set of test
dependencies and run them. This would ensure that the binary packages
were built without the test packages even present (a huge benefit, in my
opinion), but that tests were still performed.
I'm not sure how many existing tests require the built programs to still
have the build environment present (i.e. vestiges of the build process
are used by the test), but it would be nice if the binary package could
be sent to another machine where the binary package (without the source
package) was installed with the desired test packages, so the tests
could be run on the properly installed binary package.
This would, of course, require a moderate amount of implementation in
the buildds and core tools such as dpkg, and I am not in a position to
help with this, so the people who can do so would have to like my idea
enough to spend their own time to implement it.
> I think blindly applying a rule without thinking of any consequences is bad too.
I don't think anyone is blindly applying the DFSG to the source package
here. It is important to separate "DFSG Free" (main) from "Free but not
DFSG Free" (non-free) in all aspects of the source package, including
tests. The problem is that there is currently no mechanism available to
have the build process identify and select either free or non-free
tests, so only DFSG Free tests are allowed.
> I think a nocheck build profile which excludes these files from build
> is sufficient...
Only if the non-free parts of the tests are moved from the free source
package to a separate non-free source package, which still requires
something like my suggestion above to allow the non-free tests to be run
on the free binaries.
> The current policy is not making Debian better.
I disagree.
...Marvin
Reply to: