Re: Building Debian source packages reproducibly (was: Re: [RFC] Proposal for new source format)
Hello,
In fact what can be important is problem of downloading artifacts
during build. At least in Java world given application can be small but
be dependant on many libs which are downloaded during build. Program
works, build is reproducible, but we can have no idea what it consist
of.
Best regards,
Marek Mosiewicz
W dniu pon, 28.10.2019 o godzinie 10∶05 +0100, użytkownik Didier 'OdyX'
Raboud napisał:
> Le mercredi, 23 octobre 2019, 15.49:11 h CET Theodore Y. Ts'o a écrit
> :
> > On Wed, Oct 23, 2019 at 11:18:24AM +1000, Russell Stuart wrote:
> > > On Tue, 2019-10-22 at 16:52 -0700, Russ Allbery wrote:
> > > > That seems excessively pessimistic. What about Git makes you
> > > > think
> > > > it's impossible to create a reproducible source package?
> > >
> > > Has it been done? Given this point has been raised several times
> > > before if it hasn't been done by now I think it's reasonable to
> > > assume
> > > it's difficult, and thinking that it's so is not excessively
> > > pessimistic.
> >
> > Generating a reproducible source package given a particuar git
> > commit
> > is trivial. All you have to do is use "git archive". For example:
>
> When talking about upstream projects, sure.
>
> But generating Debian source packages (.dsc and friends) from a
> `debian/master` (+ `pristine-tar`) reproducibly is not really, right?
>
> As far as I understand, `gbp buildpackage -S` is the closest we have,
> but so
> far, I fail to get it to give me the bit-by-bit identical unsigned
> .dsc that
> I'd like to get. What am I missing?
>
> (A little digresssion…)
>
> Where I'm coming from is that we were discussing the tag2upload
> problem at
> miniDebConf Vaumarcus. The heart of the problem is that FTP-Master
> are
> (currently) not going to accept .dscs built reproducibly by a (even
> trusted)
> service. tag2upload is built on the idea that a signed git tag is the
> only
> needed thing (`git tag -s`) to trigger an upload, and that is not
> going to fly
> currently.
>
> The solution that seemed obvious during the discussion [0] is to
> instead rely
> on a local tool to produce a git tag with significantly more metadata
> (such as
> .dsc signature, _source.changes signature); and reconstruct the a
> signed set
> of .dsc and _source.changes automatically (as last pipeline step in
> Gitlab
> CI), which are then acceptable by the archive.
>
> In other words, its "tag2upload", but with a reproducible way to:
> - build a source package on developer machine;
> - sign it locally;
> - create and push a special git tag
> Then, in a different environment (such as a GitLab CI pipeline step),
> given a
> special git tag and a repository;
> - build the exact unsigned same source package
> - unpack the special git tag;
> - apply the signatures to get the exact same signed source packages;
> - dput to the archive.
>
> The hard part is not the packing and unpacking of the special tag;
> that's
> mostly just strings massaging. But building the exact same source
> package in
> different environments is harder than I expected.
>
> Some caveats:
> - Q: if you built and signed the source package locally, why not
> uploading?
> A: Because you might want to only upload _after_ automated tests,
> and in an
> unsupervised manner.
> - Q: If one can fit pgp signatures in a git tag; why not inlining the
> complete
> .dsc and _source.changes?
> A: Indeed. You still need the debian.tar though.
> - Q: What about Dgit: in the .dsc, or buildinfo files?
> A: Currently optional; could just be left out for a prototype.
>
> Of course, all of this can only work if we can have, or make the
> ".git to
> .dsc" conversion reproducible; hence my query.
>
> All-in-all; would this be a welcome mechanism?
>
>
> OdyX
>
> [0] It probably was already considered.
Reply to: