[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Building Debian source packages reproducibly (was: Re: [RFC] Proposal for new source format)

Le mercredi, 23 octobre 2019, 15.49:11 h CET Theodore Y. Ts'o a écrit :
> On Wed, Oct 23, 2019 at 11:18:24AM +1000, Russell Stuart wrote:
> > On Tue, 2019-10-22 at 16:52 -0700, Russ Allbery wrote:
> > > That seems excessively pessimistic.  What about Git makes you think
> > > it's impossible to create a reproducible source package?
> > 
> > Has it been done?  Given this point has been raised several times
> > before if it hasn't been done by now I think it's reasonable to assume
> > it's difficult, and thinking that it's so is not excessively
> > pessimistic.
> Generating a reproducible source package given a particuar git commit
> is trivial.  All you have to do is use "git archive".  For example:

When talking about upstream projects, sure.

But generating Debian source packages (.dsc and friends) from a
`debian/master` (+ `pristine-tar`) reproducibly is not really, right?

As far as I understand, `gbp buildpackage -S` is the closest we have, but so 
far, I fail to get it to give me the bit-by-bit identical unsigned .dsc that 
I'd like to get. What am I missing?

(A little digresssion…)

Where I'm coming from is that we were discussing the tag2upload problem at 
miniDebConf Vaumarcus. The heart of the problem is that FTP-Master are 
(currently) not going to accept .dscs built reproducibly by a (even trusted) 
service. tag2upload is built on the idea that a signed git tag is the only 
needed thing (`git tag -s`) to trigger an upload, and that is not going to fly 

The solution that seemed obvious during the discussion [0] is to instead rely 
on a local tool to produce a git tag with significantly more metadata (such as 
.dsc signature, _source.changes signature); and reconstruct the a signed set 
of .dsc and _source.changes automatically (as last pipeline step in Gitlab 
CI), which are then acceptable by the archive.

In other words, its "tag2upload", but with a reproducible way to:
- build a source package on developer machine;
- sign it locally;
- create and push a special git tag
Then, in a different environment (such as a GitLab CI pipeline step), given a 
special git tag and a repository;
- build the exact unsigned same source package
- unpack the special git tag;
- apply the signatures to get the exact same signed source packages;
- dput to the archive.

The hard part is not the packing and unpacking of the special tag; that's 
mostly just strings massaging. But building the exact same source package in 
different environments is harder than I expected.

Some caveats:
- Q: if you built and signed the source package locally, why not uploading?  
  A: Because you might want to only upload _after_ automated tests, and in an 
     unsupervised manner.
- Q: If one can fit pgp signatures in a git tag; why not inlining the complete 
     .dsc and _source.changes?
  A: Indeed. You still need the debian.tar though.
- Q: What about Dgit: in the .dsc, or buildinfo files?
  A: Currently optional; could just be left out for a prototype.

Of course, all of this can only work if we can have, or make the ".git to 
.dsc" conversion reproducible; hence my query.

All-in-all; would this be a welcome mechanism?


[0] It probably was already considered.

Attachment: signature.asc
Description: This is a digitally signed message part.

Reply to: