Re: MBF: Packages which FTBFS with the nocheck build profile
On Tue, Sep 30, 2025 at 06:09:39PM +0200, Helmut Grohne wrote:
> Would you be able to occasionally do a more complex test? Occasionally,
> the package actually cross builds with the nocheck build profile but
> misses stuff. This is not technically a FTBFS, but it also violates the
> definition of the nocheck build profile.
>
> Here is a sketch of how to test this:
>
> 1. Perform a nocheck build
> + failure? -> stop + report nocheck FTBFS
Remark: I only report a nocheck FTBFS when the package builds ok
normally, because otherwise you have to compare two different modes of
failure, and that makes things a lot more complex for little gain.
So, I know for sure that the current MBF I just did misses a few cases.
I consider those to be "hidden" by their current regular FTBFS issue,
and they will stop to be hidden when their regular FTBFS issue is fixed.
> 2. Perform a normal build
> + failure? -> stop + report FTBFS
> 3. Compare the binary artifacts of these builds for equality
> + equal? -> stop
> 4. Perform another normal build
> + failure? -> stop + report random FTBFS
> 5. Compare the binary artifacts of the two normal builds
> + equal? -> stop + report nocheck changing the result
> 6. Compare the content filenames of the nocheck build and the normal
> build
> + inequal? -> stop + report nocheck changing the result
>
> What do you think?
I have to choose my battles because I have too many of them...
I'm not ready to do cross-builds yet, and my framework is not ready to
compare binary artifacts as such (because the outcome is always "build
logs"), but maybe I should think about extracting the Checksums-Sha256
from the build logs (regular build) and do something with them,
as a sub-product of my current archive rebuilds.
Is there an easy way to get a list of packages known to be reproducible?
[ In the worst case, I could just get all the packages in sid (from
Sources.xz) and substract the ones having an entry in notes.git ]
Thanks.
Reply to: