Re: What makes a .changes file source-only? [and 1 more messages]
Guillem Jover writes ("Re: What makes a .changes file source-only? [and 1 more messages]"):
> On Wed, 2017-05-24 at 11:33:15 +0100, Ian Jackson wrote:
> > Indeed IMO it is a defect of our overall design that it the concept of
> > a `non-reproducible source package' even exists. Sources are the
> > input to builds, not an output, so the question of reproducing them
> > does not arise. That our system as a whole can sometimes mutate the
> > source package all by itself is a bug.
>
> Actually I don't think that's entirely accurate, at least for dpkg PoV.
> For non-native packages the input is the orig.tar(s) + the unpacked source
> tree, for native packages the input is just the unpacked source tree.
>
> In both cases the full Debian source packages is part of the output.
When I said "Sources are the input to builds, not an output" I was
making a general assertion about fundamentals of software engineering.
That "sources are the input to builds" is part of the definition of
"sources". Another part of the definition is that the source can be
transported conveniently from one place to another; and another is
that it is human-editable.
Insofar as any ecosystem or tool lacks something which
* is the primary input to builds
* can be easily transported
* is human-editable (with appropriate tools)
then that ecoysystem or tool is wrongheaded because software using
it does not properly has source code. (Examples might include
some kinds of persistent VM systems, ancient and modern.)
In the Debian context, the source code can only mean the source
package. The source package meets all of the above. The form you
propose is not readily transportable. (Indeed there are directory
trees which are unrepresentable as source packages.)
Insofar as dpkg's behaviour is incompatible with this view, dpkg is
buggy. As the original designer I don't think, however, that dpkg's
fundamental design is at variance with the above principles.
> > I think what you mean here is that one might have a source package
> > which is not a fixed point under `debian/rules clean' for all
> > reasonable combinations of the build-deps. I think this is a buggy
> > package but in practice it seems that many packages are buggy in this
> > way.
>
> Most probably, we'd need to check specific instances I guess. In any
> case that's one of the reasons for the .buildinfo file, so that you can
> reproduce the source package from the specific set of Build-Depends.
As I say, I think this is completely wrongheaded.
No-one cares whethere a source package "can be reproduced".
Any tools which care about reproducibility (and indeed, almost all of
our actual build infrastructure etc.) treat the source package as the
input.
Certainly, actual source packages uploaded by dgit are affected by any
infelicities that may exist in the package clean target.
> > So I think for `dgit push-source', there should be no .buildinfo ?
> > At least, unless dgit ran the clean target.
>
> The .buildinfo file on source-only uploads serves several purposes,
> one is for the reproducible source part, the other is to possibly
> include references to binary packages built but not included in the
> upload (f.ex. with «dpkg-buildpackage --changes-option=-S»).
I am only talking about truly source-only uploads, where no binary
packages were generated. See my responses to dkg.
> Not including the .buildinfo file in all (new) source-only uploads
> seems to me would make those then less uniform, and slightly more
> difficult to try to attest if they have been tampered with, as there's
> then no common advisory base-line for the environment it was built on.
Source packages are not built from objects that anyone other than the
uploader has. So it does not make sense to talk of anyone "verifying"
whether the source package was "tampered" with as it was "built".
In Debian, source packages are transported unmodified by all of our
central software including the archive and buildds.
A .buildinfo is of no help against surreptitious modification of the
source on an uploader's machine since such modifications would be to
the package source code and the .buildinfo would not help detect them.
The defences are publication, review, debdiff, etc.
> > This suggests to me that dgit push-source should use dpkg-source
> > rather than dpkg-buildpackage, as you note in later in the FAQ entry:
> >
> > If the intention is to just produce a source package instead of an
> > actual build to upload, then using dpkg-source is always the better
> > option.
> >
> > This wording is a bit unclear. It conflates `build' and `for upload'.
> > I think regarding `dgit push-source' as a build is perverse.
> >
> > dgit would have to run dpkg-genchanges.
>
> Hmm, I guess a problem might be with the overloaded meanings of build?
>
> Of course you build a source with «dpkg-source --build», in the same
> way you build a binary with «dpkg-deb --build», but doing “a build”
> in my mind would be the equivalent of preparing a release, which might
> include sources and/or binaries from «debian/rules binary» or similar,
> from dpkg-buildpackage or an equivalent tool. And whether that is
> intended as an upload would be determined by whether you have generated
> the .changes file.
Yes. In my quote above I meant `build' in the second sense. I read
`build' in your FAQ entry the same way.
My point is that uploads might not involve builds{2}. Specifically, a
pure source-only upload does not build any binaries. It is a copy
operation (with significant semantics associated with the destination
of the copy).
So it is wrong to oppose
just produce a source package
with
an actual build to upload
A pure source-only upload is "produce a source package, for upload,
and upload it". It is neither of the above, because it is not "just
produce a source package" (the package gets uploaded) and it is not
"an actual build" (since nothing is built{2}).
> > Alternatively dgit could strip out the .buildinfo, depending on
> > whether it ran rules clean.
>
> I'm not sure why that would be desirable though?
For dgit push-source, the .buildinfo at the very least misleading
(since it may well contain information about the developer's dirty
working environment, rather than any clean environment they use for
binary builds - if any) and useless (no-one benefits from it).
It is also a privacy leak.
And, dgit push-source wants to check that what it is uploading is
actually a binaryless upload. Therefore it needs to check that
binaries are not include; therefore it needs to iterate over the
.changes, and explicitly make a decision about .buildinfo.
My current view is that rejecting the upload would have been correct,
except that it is necessary to work around the bug that dpkg-source
and dpkg-genchanges generate a buildinfo when there has been no
build{2}.
Ian.
Reply to: