Hi Release Team colleagues,While tests.reproducible-builds.org has been doing a great job in bootstrapping the reproducibility problem in Debian, it does so by building twice in a row with different build environments. What we are actually interested in is if the binaries that we ship can be reproduced. Because of that, I haven't enabled the ReproduciblePolicy in britney2 to influence the migration yet. However, recently reproduce.debian.net [1] has started to try to rebuild the packages as built on the buildds. As the data from there is now (nearly) usable, I'd like to discuss whether my plan aligns with your views.
My ambition: packages in forky must build reproducible. My plan ======= 1.a Switch the ReproduciblePolicy in britney2 to the reproduce.d.n. API1.b There's one delta in the new version of the policy with respect to the current version: I didn't implement checking for regressions, see "Exceptions" below.
1.c Start with checking arch:all and arch:amd64 binaries. Currently the other architectures are not yet fully tested. I believe this is just work in progress to setup and assign workers, but until their number of unknown binaries gets close to zero, I don't think it's useful to add them. My expectation is this is a matter of days or weeks, not months.
1.d show reproducible status "for info" in the excuses, as we currently do.2. Once we've convinced ourselves that the service is fast and reliable enough and the last structural sources have been dealt with (see "standing issues" below), I want to switch to penalties for packages that ship binaries that are not reproducible. I propose we start with 10 days for this. I liked what we did during the introduction of AutopkgtestPolicy: slowly increase the penalty over time. Maybe something like 1 day extra per week?
3. Once we've got the hang of this, block the migration of packages that build non-reproducible binaries.
Exceptions ==========Obviously there are binaries that currently are not reproducible. Some statistics can be seen on the front page of reproduce.d.n [1] (showing forky). My intention is to create and maintain a list of (unversioned) hints for source or binary packages that are not reproducible in testing once we enable penalties. I'll be filing bugs at that time to warn these packages of their status (first at severity important, to be raised to serious later on). I expect there will be packages that we can't miss and will be hard to make reproducible, we can handle that via hints as long as we must.
The new version of the policy checks the source on a per binary/arch level and the hints can be either per binary or per source and can be architectured.
Standing issues ===============Currently there are a bunch of failure modes that are tracked on the statistics pages of r.d.n. I'll go over the problematic ones:
# buildinfo file 404 (maybe temporary)There is some delay in getting the buildinfo files from the buildd to buildinfos.debian.net and exposed there. A retry should solve this.
# packages missing on metasnap (maybe temporary, #1096129)Bug 1096129: when a package builds multiple versions within one dinstall run, then the first version doesn't end up on snapshot.d.o and hence is not available for rebuilding if something happened to build against that version. (MR against dak available) Also there is some delay between data available on snapshot.d.o and metasnap (which handles mapping from binary names to snapshot timestamps IIUC). A retry should solve this.
# failed to reproduce: dh-buildinfo (#1068809) Should be solved once the haskell ecosystem does its next mass upload. # failed to reproduce: dh-r (#1089197) # dpkg-buildpackage failed: dh-r (#1089197)Any upload to unstable of affected packages should be automatically fixed, so this isn't a problem for packages that try to migrate.
# failed to reproduce This is the big chunk that needs (package by package) attention. Looking forward to your response. Paul [1] https://reproduce.debian.net/
Attachment:
OpenPGP_signature.asc
Description: OpenPGP digital signature