Hi, On Sonntag, 24. Mai 2015, Stuart Prescott wrote: > > The check does not impose anything on package maintainers, like > > migration blocking or similar. So even if there was a substantial > > amount of packages that would fail that test, it would still be useful > > information for the reproducible effort IMO. > Except that reproducibility is boolean valued and this boolean is exposed > to maintainers. There is either a nice little tick on the maintainer's QA > page or there is a nasty little cross. If nasty little crosses come from > what is considered to be poor tests generating incorrect results, history > tells us that the entire column will be ignored. That makes including > tests for which there is not broad agreement a net loss for each > maintainer, for reproducible.d.n, and for the project. I agree with Stuart here. > [1] historical footnote: 10ish years ago, lintian had a reputation for > producing what were considered by many maintainers to be spurious > complaints. The result was that they didn't even bother running it at all > and so would also not see its legitimate complaints. I recall experienced > maintainers at the time telling me that lintian was only suitable for > hello world packages not for anything real. The signal:noise in that QA > tool was poor and so it became devalued; it took many years of hard work > to break down that reputation. That's exactly what I would like to avoid. On Sonntag, 24. Mai 2015, Helmut Grohne wrote: > Exactly. The only way to know whether this is a good change to policy is > to try. If it happens to make lots of packages unreproducible, then we > learned something and can revert to not varying nocheck. > > So give it a try and consider revert if it breaks the primary > reproducible goal. why don't you give it a try and test this on say 500 packages first? I don't really see reproducible.debian.net as something to experiment (much) anymore. It should producible stable results. Also: the variations we should test for are IMO those which are likely to occur *and* cannot be reasonably forced on our users. As an example: I dont think it would be reasonable to expect everybody builds in the same timezone or with the same locale or CPU type. OTOH I don't think it's totally unreasonable to say the same DEB_BUILD_OPTIONS must be used. In this sense I also think it's reasonable that we (currently) require the exact same dependencies to be installed and the build path to be identical. This is definitly suboptimal but nothing I would change at this moment as these *are* totally reasonable requirements for reproducing identical packages. I'm all for allowing achieving reproducibility in more varying environments, I just don't think that r.d.n is neccessarily the testbed for that. > Otherwise, profit from quicker testing. I assume the speed gain is almost unnoticable on average. Also, what reproducible.d.n mostly lacks are human ressources, to deal with the results. Testing for more cornercases distracts us, thus harms the project. cheers, Holger P.S.: thanks for reminding me that we should get reproducibility as a SHOULD directive in policy in 2015 ;-) In the long term I see it as a MUST directive but currently I don't even hope this will happen this year ;)
Attachment:
signature.asc
Description: This is a digitally signed message part.