Re: Not running tests because tests miss source code is not useful
On 10/8/21 7:51 PM, Pirate Praveen wrote:
> Draft text for the GR (this is my first time proposing a GR text, so
> help is welcome to make the text clearer).
>
> We should not worry about minified files in main that are (a) only in
> the source package and not in any binary package, and (b) only used to
> run tests, not to build the binary packages.
>
> To ensure these are not used during generation of binary packages, a
> nocheck build profile should be provided which will remove these files
> from the build environment.
>
> Rationale: Most of the time these minified files create a specific
> instance of a test case and these tests are not the only way to ensure
> functionality of the code we ship. Even though upstream runs these
> tests, during transitions we deviate from upstream for runtime
> dependencies and a way to ensure the functionality is not broken by a
> dependency change is essential. Insisting these test cases are built
> using only packages in main just makes running tests harder and disabing
> tests is choosen as an easier option. Treating the code used to run only
> tests is the same as generated code we ship is not useful. In case of
> the code we ship missing sources, our users can't modify the software.
> But if these sourceless code is used during tests, only ability to
> modify the test cases is lost. A test case is an arbitrary combination
> of the code and we have many different ways of testing a code.
>
> Option 1: Agree, we should allow these files in main
> Option 2: Disagree, we should not allow these files in main
> Option 3: Further discussion
>
>
A few problems with what you wrote above...
* Please don't use the word "minified" which is too much in the
Javascript world. Please use "binary object without source code"
instead, so it's more generic to any language.
* The "rationale" has nothing to do in there, if you're just attempting
to make a policy change.
* What you're proposing is clearly in strong opposition to the DFSG and
social contract, so it's IMO invalid if you're not proposing a change
over there as well to include your exception. Please propose such change
in your GR.
> But if these sourceless code is used during tests, only ability to
> modify the test cases is lost.
I strongly do not agree with the sentence above: in the way it's
constructed, in how it's formulated, and with the consequences.
I am convince that, without running a test case, it's not reasonable to
even attempt to modify a piece source code. Tests are written because
it's easy to break some code. Not having access to the test case makes
it very hard to understand how it works, and how the software is
actually built, and potentially hacked/changed.
> A test case is an arbitrary combination of the code and we have many
> different ways of testing a code.
A test case is not "an arbitrary combination of the code", in a TDD way,
it's often written before the rest of the code, and it is necessary in
order to avoid breakage. If you're arguing that tests could be rewritten
by anyone, then yeah: do it and ship the rewritten fully-free-software
test then, if it's that easy... This is rhetorical: both you and me know
it's not the case that it is easy to rewrite a full set of tests. It
takes a lot of time too.
Cheers,
Thomas Goirand (zigo)
Reply to: