[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: MBF: Packages which FTBFS with the nocheck build profile



On Tue, Sep 30, 2025 at 06:09:39PM +0200, Helmut Grohne wrote:

> Would you be able to occasionally do a more complex test? Occasionally,
> the package actually cross builds with the nocheck build profile but
> misses stuff. This is not technically a FTBFS, but it also violates the
> definition of the nocheck build profile.
> 
> Here is a sketch of how to test this:
> 
>    1. Perform a nocheck build
>       + failure? -> stop + report nocheck FTBFS

Remark: I only report a nocheck FTBFS when the package builds ok
normally, because otherwise you have to compare two different modes of
failure, and that makes things a lot more complex for little gain.

So, I know for sure that the current MBF I just did misses a few cases.
I consider those to be "hidden" by their current regular FTBFS issue,
and they will stop to be hidden when their regular FTBFS issue is fixed.

>    2. Perform a normal build
>       + failure? -> stop + report FTBFS
>    3. Compare the binary artifacts of these builds for equality
>       + equal? -> stop
>    4. Perform another normal build
>       + failure? -> stop + report random FTBFS
>    5. Compare the binary artifacts of the two normal builds
>       + equal? -> stop + report nocheck changing the result
>    6. Compare the content filenames of the nocheck build and the normal
>       build
>       + inequal? -> stop + report nocheck changing the result
> 
> What do you think?

I have to choose my battles because I have too many of them...

I'm not ready to do cross-builds yet, and my framework is not ready to
compare binary artifacts as such (because the outcome is always "build
logs"), but maybe I should think about extracting the Checksums-Sha256
from the build logs (regular build) and do something with them,
as a sub-product of my current archive rebuilds.

Is there an easy way to get a list of packages known to be reproducible?
[ In the worst case, I could just get all the packages in sid (from
Sources.xz) and substract the ones having an entry in notes.git ]

Thanks.


Reply to: