Bug#1007717: Native source package format with non-native version
A friend suggested some information that I could usefully provide:
Firstly, I posted to -devel a summary of why format 3.0 is not
pareto-better than 1.0. I have c&p it below.
Secondly, I was asked if there was a bug against src:dpkg asking for
"3.0 (native)" to allow packages with Debian revisions. I don't think
there is - at least, I didn't find one. I do remember this coming up
before, but I don't seem to be able to find a record of it now.
Sean Whitton writes ("Re: Bug#1007717: Native source package format with non-native version"):
> I think that you are asking us to use 6.1.1/6.1.2 for most of this, but
> for I.3 are you asking for us to use 6.1.4 or 6.1.5? "Request" seems
> more like 6.1.5 but I would like to check.
I think you mean "for I.2 are you asking". I am not asking for the TC
to overrule the dpkg maintainer under 6.1.4.
Let me requote myself with annotations:
> On Tue 15 Mar 2022 at 04:29pm GMT, Ian Jackson wrote:
> > Please:
> >
> > Part I - belss continued use of 1.0 native format, for now at least:
> >
> > 1. Declare explicitly that there is nothing wrong with a package with
> > a native format, but a non-native version number.
6.1(1) Decide on any matter of technical policy.
> > 2. Request that the dpkg maintainer relax the restriction which
> > prevents the use of 3.0 native with Debian revision.
6.1(5) Offer advice.
> > 3. Consequently, declare that the recent MBF on this topic ought not
> > to have been filed against packages where simply changing the
> > source format does not currently work. That would include at
> > least 1.0 native packages with Debian revisions.
6.1(5) Offer advice.
> > Part II - bless continued use of 1.0-with-diff, for now at least:
> >
> > 4. Declare that sometimes the use of 1.0-with-diff can be the best
> > tradeoff between different considerations. In particular,
> > because 1.0 is the only format which botH:
> > (a) Optimises bandwidth and storage by reusing the upstream
> > data when it hasn't changed.
> > (b) Avoids polluting the working tree (package source code)
> > with [patches], which cause trouble especially with
> > git-based workflows.
6.1(5) Offer advice.
> > 5. Consequently, declare that the recent MBF on this topic ought not
> > to have been filed against 1.0 with diff packages, at least
> > without some further filter.
6.1(5) Offer advice.
Thanks,
Ian.
From: Ian Jackson <ijackson@chiark.greenend.org.uk>
To: Sean Whitton <spwhitton@spwhitton.name>
Cc: Russ Allbery <rra@debian.org>,
debian-devel@lists.debian.org
Subject: Re: proposed MBF: packages still using source format 1.0
Date: Wed, 9 Mar 2022 16:38:43 +0000
Sean Whitton writes ("Re: proposed MBF: packages still using source format 1.0"):
> On Wed 09 Mar 2022 at 01:08pm +01, Lucas Nussbaum wrote:
> > On 08/03/22 at 17:33 -0700, Sean Whitton wrote:
> >> Lucas, as I've had a lot to do with these git workflows and have
> >> probably done the most work documenting them, I can help with any
> >> specific follow-up questions you might have.
> >
> > Thanks!
> >
> > So the main question I think I have is:
> >
> > can we find a middleground where the git workflows don't require staying
> > with 1.0? Even if that means switching to 3.0 (quilt) using the
> > single-debian-patch approach?
>
> dgit-maint-merge(7) works with single-debian-patch and that's what I
> use. But it is not clear to me that there are any advantages at all to
> that over 3.0 (quilt) with single-debian-patch?
The situation here is complicated.
The tl;dr is that
* there are several situations where 1.0-native is the best answer,
* there are several situations where 1.0-with-diff is the best answer,
The root cause of both of these situations is that 3.0, sadly,
is not always better in every respect than 1.0.
1. Why is 1.0-without-diff not always worse than 3.0 (native) ?
1.0 native is sometimes better than 3.0 (native) because dpkg-source
refuses to build a 3.0 native package with a Debian revision in its
version number.
This prohibition exists solely because of a doctrinal objection to
native-format packages with Debian revisions. There is no technical
reason why this restriction could not be lifted. I sometimes upload
this way and I have never had anyone report problems[1] with it.
IMO there is nothing wrong with native format packages with Debian
revisions. They work just fine. For a small paockage, this is often
a good choice, because it avoids dealing with patches at all.
For anything but a small package, use of diffs is needed as a storage
and distribution optimisation.
2. Why is 1.0-with-diff not always worse than some 3.0 format ?
1.0-with-diff has the following advantage over 3.0 (quilt):
The extracted source tree does not contain a diff. The inclusion of
the diff *inside* the source tree (which happens with "3.0 (quilt)"
whether or not single-debian-patch is specified) causes all manner of
problems: it means that only certain states of the extracted tree are
valid.
With 3.0 and single-debian-patch, modifying ordinary files can
generate a tree which doesn't round-trip through dpkg-source --build
and dpkg-source --extract: dpkg-source --build needs to "commit" the
changes to the diff, amending the working tree.
In other words, the working tree contains output files. This causes
trouble if you want to represent the package in git. This is why dgit
has all this quilt-fixup stuff. With 1.0-with-diff, no quilt fixup is
needed.
3. Why is 1.0-native not just as good as 3.0 (native) ?
The only significant advantage of 3.0 (native) is that dpkg-source is
willing to create and extract 3.0 packages using newer compression
algorithms such as xz.
There were no good reason why the additional compression algorithms
were not supported in 1.0. ("3.0 (native)" does offer some minor
metadata format advantages, so its existence is not entirely
pointless.)
If the restriction against native format with Debian revision were
lifted, "3.0 (native)" would be as capable as 1.0-without-diff, and
slightly superior. So in that case, "3.0 (native)" should probably
replace all uses of 1.0-native.
4. What disadvantages does 1.0-with-diff suffor from, compared to "3.0
(quilt)" with single-debian-patch ?
The main disadvantage is that there are changes to the source tree
which are representable in "3.0 (quilt)" but which 1.0-with-diff does
not support. (The precise details and the history are too compiex to
go into.)
I haven't checked recently, but this used to be enforced both on
building and on extraction. So changing this is not so simple, since
we shouldn't start distributing packages in a new source format (or
variant) until the new extraction capability is very widely deployed.
But, ultimately, it would probably be a good idea. Whether the to
call resulting thing "1.0" or "3.0 (diff)" doesn't really seem very
important.
The important properties are that it should support at least every
change that current diff(1) can represent.
5. Why is native sometimes superior to any quilt or diff format
We rely on diff(1) to represent changes to source trees and patch(1)
to apply them. Not every change is representable by diff.
diff/patch does improve, of course. That is not an unalloyed benefit,
though, because it means that when diff/patch improve we can
accidentally generate source packages that earlier versions can't
extract - a compatibility hazard.
Changes not representable by diff is what Sean is talking about here:
> Ian has some cases where something that is representable in git is not
> representable using 3.0 (quilt) but is representable using 1.0. I don't
> have those cases to hand; Ian, could you summarise, please?
Currently, I think diff cannot represent changes to symlinks.
git can store symlinks and represent their targets, and changes to
their targets.
Ian.
[1] By "problems" I mean "some software did a wrong thing", not
"I am offended by your breach of my notions of propriety".
--
Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Reply to: