Re: What makes a .changes file source-only? [and 1 more messages]
[ Sorry for the delay, got caught in other stuff, release, real life, etc,
and the thread looked like requiring some non-small blocks of time. ]
Hi!
On Wed, 2017-05-24 at 11:33:15 +0100, Ian Jackson wrote:
> Guillem Jover writes ("Re: What makes a .changes file source-only?"):
> > Of course, in practice, if going just by the Architecture field, you
> > need to trust that the software generating the .changes (and the .dsc)
> > is not buggy, and the entity that commissioned its creation is not
> > trying to bypass the checks. But for BY-HAND artifacts that do not
> > follow the well defined name_version_arch.type filename form, then
> > this will not be represented in the Architecture field, which is
> > something that should probably be fixed by annotating the field with
> > some value (probably the host architecture to be conservative).
>
> I think this is a bug, then, in dpkg. If Architecture is `source',
> then there shuld not be any by-hand artifacts.
Actually, nevermind, this might have just been a regression during the
1.18.x cycle, but starting with the 1.18.19 release, dpkg-genchanges
should not add any artifact that is not a source nor a buildinfo file
for source-only uploads.
> > Also, even though I could imagine someone injecting non-source artifacts
> > from within the debian/rules clean target even for source only builds,
> > I'd consider that to be just broken.
>
> It's rather weird that a source package build still looks at the
> debian/files.
The debian/files is just used to be able to communicate the .buildinfo
filename from dpkg-genbuildinfo to dpkg-genchanges.
> > For source-only, I think going by a whitelist is indeed more sensible,
> > but I'd just check whether there is a .dsc, and whether the rest of the
> > references in the .changes are just the files referenced in the .dsc.
>
> I like this suggestion. Thanks.
>
> But:
>
> > > We observed that .buildinfo files are included in purportedly
> > > source-only changes files by `dpkg-buildpackage -S`.
> > >
> > > Is this correct? Why are they included in source-only uploads?
> >
> > Yes, this is correct, although I've noticed again (as I did in the
> > past but seem to have forgotten) that the dpkg-buildpackage man page
> > is out-of-sync regarding this, which I'll be fixing. In any case the
> > other day I just added a FAQ entry, given that this seems a recurring
> > question. :)
> >
> > <https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Why_are_.buildinfo_files_always_generated_with_dpkg-buildpackage.3F>
> >
> > If there's anything that does not seem to hold, or is unclear I'm happy
> > to clarify it further.
>
> Well, of course, your suggestion to check the .dsc against the
> .changes will trip over the .buildinfo.
Sorry for the confusing sentence, yes, I meant to say the .dsc and the
.buildinfo files here.
> Also, as you write in your FAQ entry:
>
> By default dpkg-buildpackage does active tasks such as cleaning via
> debian/rules, and makes sure that the dependencies from Build-Depends
> are satisfied as these are needed by the clean target. In addition the
> clean target can perform any kind of action that will affect the
> source package, which is also part of the build, and should be by
> default also reproducible
> I think what you mean here is that one might have a source package
> which is not a fixed point under `debian/rules clean' for all
> reasonable combinations of the build-deps. I think this is a buggy
> package but in practice it seems that many packages are buggy in this
> way.
Most probably, we'd need to check specific instances I guess. In any
case that's one of the reasons for the .buildinfo file, so that you can
reproduce the source package from the specific set of Build-Depends.
Also, I don't remember where, but ISTR some of our documentation
recommending (or used to) that some update actions should be done in
the clean target (maybe that's old and no longer the case though).
> Indeed IMO it is a defect of our overall design that it the concept of
> a `non-reproducible source package' even exists. Sources are the
> input to builds, not an output, so the question of reproducing them
> does not arise. That our system as a whole can sometimes mutate the
> source package all by itself is a bug.
Actually I don't think that's entirely accurate, at least for dpkg PoV.
For non-native packages the input is the orig.tar(s) + the unpacked source
tree, for native packages the input is just the unpacked source tree.
In both cases the full Debian source packages is part of the output.
I do agree that having a non-declarative active actions when doing
source-only uploads is an issue. And it might have been better to
instead have a declarative one, that some dpkg-foo tool would use to
prepare the tree when building the source package.
> That is, even with the .buildinfo, someone who gets the .dsc cannot
> know whether the rules clean target is correct (or to put it another
> way, under what conditions the source tree is a fixed point under
> rules clean), because dgit has not necessarily run rules clean at all.
> I'm sure there are other vcsish tools which have the same property.
>
> (Also, and I hesitate to make this argument because of course I
> support reproducible builds, but: if the .buildinfo is not useful,
> then it's an unwarranted privacy violation.)
The .buildinfo file contains at most advisory information. If the
entity producing the upload is malicious then they have control over
its contents, so you'd need to build it and compare the only thing
you've possibly got in front of you, which are the actual artifacts.
If the entity producing the upload is not malicious it might still
be running a buggy toolchain, or on a broken/damaged hardware or
installation, etc. So no matter what, you can use the information
in the .buildinfo to ease tracking the environment supposedly used,
but you then really need to build the stuff to check if it matches.
> So I think for `dgit push-source', there should be no .buildinfo ?
> At least, unless dgit ran the clean target.
The .buildinfo file on source-only uploads serves several purposes,
one is for the reproducible source part, the other is to possibly
include references to binary packages built but not included in the
upload (f.ex. with «dpkg-buildpackage --changes-option=-S»).
Not including the .buildinfo file in all (new) source-only uploads
seems to me would make those then less uniform, and slightly more
difficult to try to attest if they have been tampered with, as there's
then no common advisory base-line for the environment it was built on.
Of course dgit uploads are marked as such, but you could concoct that.
> This suggests to me that dgit push-source should use dpkg-source
> rather than dpkg-buildpackage, as you note in later in the FAQ entry:
>
> If the intention is to just produce a source package instead of an
> actual build to upload, then using dpkg-source is always the better
> option.
>
> This wording is a bit unclear. It conflates `build' and `for upload'.
> I think regarding `dgit push-source' as a build is perverse.
>
> dgit would have to run dpkg-genchanges.
Hmm, I guess a problem might be with the overloaded meanings of build?
Of course you build a source with «dpkg-source --build», in the same
way you build a binary with «dpkg-deb --build», but doing “a build”
in my mind would be the equivalent of preparing a release, which might
include sources and/or binaries from «debian/rules binary» or similar,
from dpkg-buildpackage or an equivalent tool. And whether that is
intended as an upload would be determined by whether you have generated
the .changes file.
Does that clarify things? It's also very possible my mental model does
not match that of other people. :)
Anything that needs to produce a .changes file is preparing a possible
upload in my mind.
> Alternatively dgit could strip out the .buildinfo, depending on
> whether it ran rules clean.
I'm not sure why that would be desirable though?
Thanks,
Guillem
Reply to: