Bug#1107137: debian-policy: Assumed need to adapt to no longer meaningful nativeness concept
Source: debian-policy
Source-Version: 4.7.2.0
Hi!
[ I'm exhausted and tired about this topic, and the previous and recent
litigiousness and confrontational vibe surrounding it all, which has
been dragging on for years. So for some of the following stuff, I
cannot be bothered to recheck or provide links and references. Sorry. ]
Summary
-------
Given the previous bogus advice the TC provided, and that I've been told
a new ruling would follow that, to try to reduce the chance of making
the surrounding code area a "toxic wasteland" with uncertainties on what
can be changed or not w/o having to consult the TC every time (which I'd
rather not interact with at all), I've committed a change to dpkg git main
that makes the «3.0 (native)» source format be "fuzzy native" for Debian
and derivatives. This will be in principle part of dpkg 1.23.0 (in Debian,
targeting forky). (This also deprecates the Dpkg::Version->is_native()
method which can no longer be portably used with consistent/coherent
semantics, to be removed at a later point.)
Where I guess this broken new notion will keep being pushed either in a
litigious way or similar (and where the «3.0 (native)» format might become
a «toxic wasteland» of development uncertainty (in Debian and derivatives)
anyway (where I assume it will not be worth modifying further). I'm
assuming (that's why this filing) the Debian policy will need to be
adapted too to this new reality. See below.
Background
----------
This has been covered on some dpkg (#700177, #737634), policy (…),
lintian (#944155) and tech-ctte bugs (…), and several threads on d-d
over the years where my position was
<https://lists.debian.org/debian-devel/2014/02/msg00102.html>
<https://lists.debian.org/debian-devel/2019/11/msg00073.html>
<https://lists.debian.org/debian-devel/2019/11/msg00081.html>.
When the consistency and coherence check was introduced for 3.0 formats,
this affected:
- 0 packages in «3.0 (quilt)» format.
- 9 packages in «3.0 (native)» format, most of which seemed like
accidental packaging mistakes.
For 1.0 format(s), this is the classification of the affected packages
over time:
- Current affected packages (10), and maintainers (6):
,---
$ Sources=$(apt-get indextargets \
--format '$(FILENAME)' 'Identifier: Sources')
$ /usr/lib/apt/apt-helper cat-file $Sources | grep-dctrl \
-n -sPackage -FFormat '1.0' -a \
-FVersion '-' -a --not -FFiles '.diff.'
chiark-tcl-applet
chroma
python3-defaults
rust-derive-deftly
spigot
sympathy
userv-utils
valgrind-if-available
xbs
xtruss
`---
- Accidental packaging mistakes:
+ Most might have been fixed due to the lintian tag introduced later,
some due to the dpkg-source warnings introduced even later, the rest
due to bugs being filed. From me for example at:
<https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=dpkg-mismatch-source-vs-version-format;users=debian-dpkg@lists.debian.org>
and #947051, but other people might have filed similar reports.
- Metapackages, where the intention is to track another upstream
versioning schema, but from a native package (which can be handled via
version remapping for src→bin pkgs), there are only a handful of these
that I'm aware of.
- Workflow "reasons":
+ In-archive use, due to a handful of maintainers, that primarily use
git based workflows, where some even consider that source packages
are a relic and should be completely abolished. There has also been
incorrect reasons given in the past such as claiming «3.0 (quilt)»
not supporting git formatted patches (those have been supported
since GNU patch(1) gained support for them, which was made a hard
requirement for dpkg).
+ Out of archive use, like in git based workflows or CI settings when
building out of git, where alternatives have been suggested in the
past, such as using «git archive», pristine-tar (which I'm not a fan
of) or pristine-lfs, or via the «3.0 (git)» or «3.0 (custom)» formats,
or even using a different «-» character instead of abusing the
revision marker, but the complainants have refused all of those
(clearly misusing the format is more convenient, in detriment to
everyone else).
<https://lists.debian.org/debian-devel/2022/03/msg00176.html>
Because the 1.0 format(s) are sloppy and their behavior is context
sensitive based on whether specifically named tarballs might be around,
or depending on the dpkg-source options given, there was an implicit
compromise on allowing this incoherence and inconsistency for that format
in its "native form" while still emitting a warning, until a new source
format, an existing one could be improved, or better tooling or workflows
could be implemented or devised. This compromise was made explicit on
the "recent" mass bug filing thread on d-d:
<https://lists.debian.org/debian-devel/2022/03/msg00285.html>
There was never a reply to the dpkg bug report after the TC involvement
(except for <https://lists.debian.org/debian-dpkg/2023/11/msg00013.html>),
because I never found the energy to engage with it afterwards, given
the litigious and contentious discussions this involved, and because
it's hard to find the motivation to, say, devise a new source format,
or improve the surrounding tooling or an existing source format, when
the vocal people that have been pushing for this, seem to either only
want to use this because their CI requires them to (and are not willing
to consider any of the proposed alternatives), or think that source
packages should be completely abolished anyway, or where for the
metapackages case this would be used by a handful of packages in the
archive at most anyway.
<https://lists.debian.org/debian-devel/2022/03/msg00173.html>
Actions
-------
In the Debian context, this pushed confusion also seems clearly
against its policy, so from the Debian policy side, I guess you'll need
to scrap or heavily reword at least most of §4, and parts of §3.2.1,
§12.7, §5.6.12, §5.6.12.2.
So, I guess you'll need to mention or document somehow, somewhere
that the native concept is fuzzy, and that, well, in Debian and
derivatives, it does not have much of a real meaning anymore going
forward.
After that, I guess someone should probably go around fixing at least
devscripts and dgit (which are using Dpkg::Version, I don't know about
other stuff using other language bindings, or using their own parsing,
etc), and anything else expecting (as it would be logical) that a native
version implies a native source package, which might need to be made
aware of source specific formats (in case that information is even
available at all).
Someone might also want to add some new questions to the NM templates
(if these are not already there), as in stuff like:
- "what is a native source package?"
- "what is a native version?"
- "can a native source package have a non-native version?"
- "can a non-native source package have a native version?"
- "why?"
Consequences
------------
I'm assuming that once this lands in dpkg in Debian, we'll start to get
new regressions, for the convenience of being able to abuse the concept
while undermining these source package formats, for the benefit of people
that seem to be using them in anger and would rather see them gone anyway,
where the existing warnings have been shown to not be enough, and where
people seem to miss them.
I'll stop tracking these recurring mistakes, as I can't be bothered
anymore.
This makes the native source concept, have incoherent and confusing
meaning/semantics, and it stops being symmetric with non-native source
packages. Where one of the usual complaints people have over the dpkg
tooling and packaging system and stack in general, and its concepts is
that they are too complex, so I guess this goes into making it even
more confusing and incoherent.
Going forward
-------------
I've also been pondering about native source for a while, and I guess this
is the last drop in the bucket, and I'm going to be strongly considering to
stop using it for packages I maintain. It does not support upstream
signatures (and downstreams not on a dpkg-based distro do not have much of
a use for signed .dsc); the NMU rules for native sources still look very
wrong to me, because they take over the "upstream" versionspace, as if it
was an official/supported release, when such NMUs should be switching to
a non-native format; in a similar vein, downstream tend to patch the native
source as is, instead of switching it to a non-native format, which makes
the delta, non-obvious, and also takes over the versionspace, and further
complicates its own downstreams which could otherwise more easily choose
to opt out of specific changes.
I'm still convinced this is wrong, sloppy, and adds confusion and
incoherence, for both tools and humans, that's why the dpkg code still
considers this an error for non-Debian and non-Debian derivative vendors
(where I've now used the opportunity to also make the 1.0 source format
case a hard error there, given that the compromise has been broken
anyway and this is vendor specific now), but for Debian and derivatives,
I cannot be bothered anymore. Someone else will need to handle any
possible fallout now, and going forward…
[ I'm very unlikely to engage further on this topic. ]
Whatever,
Guillem
Reply to: