Re: dpkg-source and gitish patches
Hi!
On Wed, 2016-01-20 at 14:27:22 +0000, Ian Jackson wrote:
> I discovered recently that wheezy cannot unpack some source packages
> that are in jessie-backports, because they contain (in their 3.0
> (quilt) stack) gitish patches that earlier versions of patch do not
> properly understand.
>
> (ISTM also possible that under other circumstances such patches might
> be partially applied (eg, only file content changes and not mode
> changes, or not renames, or something). IDK whether this is actually
> a risk.)
>
> Was this non-backwards-compatible change to the source format
> introduced deliberately ?
Yes and no.
IIRC I became aware of this issue while investigating the problem with
bug #746306 (or perhaps one of the previous CVEs), the difference with
the solution I ended up taking there to completely ban C-style strings
to represent filenames is that git patches are pretty much ubiquitous
(from people doing «git format-patch» f.ex.), and banning them would
render a significant part of the archive unbuildable. Given that patch(1)
had been natively supporting them for a while, I was not sure how
widespread the usage of the git-style extended format (not just simple
patches with the «diff --git» marker) was on the archive in general or
stable in particular. And banning just patches using the git-style
extended format seemed risky for a stable update, as it could make
things unbuildable and it would require non-insignificant code changes
to dpkg-source, or source packages.
So my conclusion was that the safest option for now was to do damage
control, make dpkg-dev require the new patch supporting git-style diffs
(commit 9e26996fa45cd5fc1c5b92025fddf3cac5c7b1a5), and consider this
in a good light as a feature that we could use in a nice way.
Maybe that was the wrong choice, and I should probably have brought this
up on the list for wider comment. If people consider this is the case,
I'm open to reverse course after a careful analysis of the current
situation in stable/testing/unstable, etc. But while I'll be happy to
handle the dpkg side, I don't think I can push any other part of such
“major” transition right now.
> AFAICT there is no way in dpkg-source to control this compatibility
> aspect: ie, there is no way to say `please generate only traditional
> format "3.0 (quilt)" packages'; nor to say `this source will not be
> transferred properly unless a gitish patch is used, so the new
> features must be available'.
>
> Do you think there should be such a mechanism ? What should it look
> like ?
Right, probably there should be one. But TBH I didn't consider this,
in a way, given that I saw it as matter of an inevitable change
inherited from patch(1), and the prevalence of git formatted patch in
the wild.
I'd be open to have an option to dpkg-source to reject such patches
for example and error out when creating a source package. Or other
proposals.
For the more generic case, we might want to make dpkg-source record
such features that it can detect in a new field in the .dsc. Of course
that covers only things that we are aware of beforehand. That's also
one of the reason I've been trying lately to make the dpkg suite in
general more strict in what it accepts, because otherwise we get this
kind of thing slipping under our radar. :/
> ISTM that the next time we make a change like this it ought to be done
> by the maintainer explicitly requesting a new source format. That is
> how non-backwards-compatible changes to source formats have been done
> in the past. A series of announcements on d-d-a with transition
> availability etc. information would be good too.
Yeah, that's my ideal. :/
OTOH there's been also recent changes that ended up banning several
existing patches due to them being somewhat bogus (although accepted
by patch(1)), that could not be discerned from malicious ones (f.ex.
commmit 5348cbc981a65c3c9b05bb4d13553bda930c2d78).
> Having said all that:
>
> Now that are relying on gitish diffs, are there any plans to make
> dpkg-source able to represent binary diffs, and file removals ?
Given that we have bitten the bullet on this (at least for now), I
was intending to bring up supporting git-style diffs natively in
dpkg-source, I've just not had the time to get to it yet. If you'd
like to work on that, that would be great.
Unfortunately last time I checked on this (see below), I realized that
the git binary diff format is not documented anywhere, so ISTM this
would need to be requested first to git upstream.
> (This is an alternative to my rsync proposal, which I have sadly put
> on the back burner as it seems quite complicated to do all the things
> everyone wants before it could be accepted. I still think `3.0
> (rsync)' would be a good format - and it would suffer from fewer
> compatibility problems, and fewer risks from immaturity of
> dependencies, than `3.0 (quilt) [gitish]')
(I've also got a half-finished draft reply to that, which I'd like
get the time to send out, at least to clarify my position on it.)
Thanks,
Guillem
Reply to: