Re: mercurial new test packages
On 2018-06-28 21:56:07, Chris Lamb wrote:
> Hey Antoine, :)
>
>> The package I managed to build obviously passes that test suite, and
>> *reliably* [but] it might FTBFS on the buildds
>
> Thanks for working on this. :)
>
> I'm a bit lost by your wording; it "might" FTBFS on the buildds, it
> does not sound particularly reliable to me… and thus doesn't sound like
> something we really want to upload. Perhaps I'm missing something here.
I am not sure why the test suite fails nor why the output varies from
one build to the next. Once a package is built, however, it passes the
test suite reliably. Therefore, I suspect this might be a weird local
problem or something else. So while I was eventually able to build it
here, it might or might not fail on the buildds. The other architectures
might succeed and so on.
I also suspect the problem was present before this new version, but I
never noticed it. I remember you had trouble with building the package
reliably before as well, so this might be a more general issue with the
package as a whole.
>> I was not able to make the package build reproducibly
>
> I was further confused by this. :) In particular why you are striving
> for reproducibility at all?
My theory is that the variations in the test suite that trigger the
failure are due to variations in the built Mercurial binaries, which
change the order of some of the answers of the `hg serve` command. I
base this on the fact that, once a package successfully built, it
reliably outputs the bits the test suite requires correctly, every
time. When you *build* the package, however, and then the test suite
runs, the output varies. This makes me think the actual binary is
somewhat different on very built.
> A reliable build — ie. what we most
> care about here — is a necessary but not sufficient condition for
> reproducibility, but I don't see that achieving a reproducible build in
> an jessie LTS upload is very useful. Again, I'm sure I'm missing
> something.
I agree: we want a reliable build that reliably passes the test suite. I
connect this with reproducibility because I think the problem lies in
the built `hg` binary. This might be incorrect, of course.
>> something in the *build* process changes the output non-
>> deterministically, which is very strange to me.
>
> (My gut tells me that this is non-deterministic filesystem ordering..)
That could very well be! When I tried to reproduce the results here, I
only created the test repository once.
So this might be something to look into...
A.
--
À force de ne jamais réfléchir, on a un bonheur stupide
- Jean Cocteau
Reply to: