Bug#542288: debian-policy: Version numbering: native packages, NMU's, and binary only uploads

[Russ Allbery <rra@debian.org>]

Here is an updated diff that documents the most well-understood version
conventions in the Debian archive.  More could certainly be added; this is
just a first start that addresses this specific bug.

This revised patch is less aggressive about defining native packages to
only mean packages with no existence external to Debian.  I also found
that we never define upstream, which seems like a critical concept, so I
added a definition to the Definitions section.

I've also reviewed the places where the Developer's Reference talks about
similar issues and I believe this is consistent with it.

I think this change is ready for seconds.

-- 
Russ Allbery (rra@debian.org)              <https://www.eyrie.org/~eagle/>

diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
index a21a510..cd7daaa 100644
--- a/policy/ch-controlfields.rst
+++ b/policy/ch-controlfields.rst
@@ -582,20 +582,17 @@ The three components here are:
     alphanumerics and the characters ``+`` ``.`` ``~`` (plus, full stop,
     tilde) and is compared in the same way as the ``upstream_version`` is.
 
-    It is optional; if it isn't present then the ``upstream_version``
-    may not contain a hyphen. This format represents the case where a
-    piece of software was written specifically to be a Debian package,
-    where the Debian package source must always be identical to the
-    pristine source and therefore no revision indication is required.
+    It is conventional to restart the ``debian_revision`` at ``1`` each
+    time the ``upstream_version`` is increased.
 
-    It is conventional to restart the ``debian_revision`` at ``1``
-    each time the ``upstream_version`` is increased.
+    The package management system will break the version number apart at
+    the last hyphen in the string (if there is one) to determine the
+    ``upstream_version`` and ``debian_revision``. The absence of a
+    ``debian_revision`` is equivalent to a ``debian_revision`` of ``0``.
 
-    The package management system will break the version number apart
-    at the last hyphen in the string (if there is one) to determine
-    the ``upstream_version`` and ``debian_revision``. The absence of a
-    ``debian_revision`` is equivalent to a ``debian_revision`` of
-    ``0``.
+    Presence of the ``debian_revision`` part indicates this package is a
+    non-native package (see :ref:`s-source-packages`).  Absence indicates
+    the package is a native package.
 
 When comparing two version numbers, first the epoch of each are
 compared, then the ``upstream_version`` if epoch is equal, and then
@@ -646,6 +643,83 @@ numbers containing strings of letters which the package management
 system cannot interpret (such as ``ALPHA`` or ``pre-``), or with silly
 orderings.  [#]_
 
+Special version conventions
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following special version numbering conventions are used in the Debian
+archive:
+
+- The absence of ``debian_revision``, and therefore of a hyphen in the
+  version number, indicates that the package is native.
+
+- ``debian_revision`` components ending in ``.`` (period) followed by a
+  number indicate this version of the non-native package was uploaded by
+  someone other than the maintainer (an NMU or non-maintainer upload).
+  This is used for a source package upload; for uploads of only binary
+  packages without source changes, see the binary NMU convention below.
+
+- ``upstream_version`` components in native packages ending in ``+nmu``
+  followed by a number indicate an NMU of a native package.  As with the
+  convention for non-native packages, this is used for a source package
+  upload, not for uploads of only binary packages without source changes.
+
+- ``upstream_version`` components in native packages or
+  ``debian_revision`` components in non-native packages ending in ``+b``
+  followed by a number indicate a binary NMU: an upload of a binary
+  package without any source changes and hence without any corresponding
+  source package upload or version change.
+
+- ``upstream_version`` components in native packages or
+  ``debian_revision`` components in non-native packages ending in
+  ``+debNuX`` indicate a stable update.  This is a version of the package
+  uploaded directly to a stable release, and the version is chosen to sort
+  before any later version of the package uploaded to Debian's unstable
+  distribution.  ``N`` is the major version number of the Debian stable
+  release to which the package was uploaded, and ``X`` is a number,
+  starting at 1, that is increased for each stable upload of this package.
+
+  For example, suppose Debian 10 released with a package with version
+  ``1.4-5``.  If that package later receives a stable update in Debian 10,
+  the first update would have the version ``1.4-5+deb10u1``.  A subsequent
+  update would have version ``1.4-5+deb10u2``.  These numbers are designed
+  to sort earlier than ``1.4-6``, the version number that would be used
+  for the next unstable upload.
+
+- ``upstream_version`` components in native packages or
+  ``debian_revision`` components in non-native packages ending in
+  ``~debNuX`` also indicate a stable update, but of a different type.
+  This version convention indicates that the stable version of the package
+  was updated to a new upstream release, as distinct from the ``+debNuX``
+  convention that indicates additional changes were applied to the
+  existing stable release.  ``N`` and ``X`` mean the same as with
+  ``+debNuX``.
+
+  For example, suppose Debian 10 released with a package with version
+  ``1.4-5``, and then suppose a new upstream release was later packaged as
+  ``1.5-1``.  In some exceptional circumstances, it may make sense to
+  replace the stable version of the package with a package based on the
+  upstream ``1.5`` release and ``1.5-1`` Debian package instead of trying
+  to apply patches to the ``1.4-5`` version.  In this case, the new stable
+  package would have version ``1.5-1~deb10u1``.  This number is designed
+  to sort earlier than ``1.5-1``, the version number that would be used
+  for the unstable upload.
+
+- ``upstream_version`` components in native packages or
+  ``debian_revision`` components in non-native packages ending in
+  ``~bpoNuX`` indicate a backport of a version of the package to an older
+  stable release.  The part of the version before ``~bpo`` is the version
+  of the package being backported, ``N`` is the major version number of
+  the Debian stable release to which the package was backported, and ``X``
+  is a number, starting at 1, that is increased for each revision of the
+  backport of that package version.
+
+  This version convention was chosen to sort before the original package
+  release that is being backported, so that the backport will upgrade to
+  the original package during a later system upgrade to a newer Debian
+  release.
+
+This list of version conventions is not exhaustive.
+
 .. _s-f-Description:
 
 ``Description``
diff --git a/policy/ch-scope.rst b/policy/ch-scope.rst
index e3db6c1..45f150f 100644
--- a/policy/ch-scope.rst
+++ b/policy/ch-scope.rst
@@ -182,6 +182,24 @@ ASCII
     first 128 `Unicode <http://www.unicode.org/>`_ characters, with the
     eighth bit always zero.
 
+upstream
+    The source of software that is being packaged, or the portion of a
+    software package that originates from outside of Debian.  For example,
+    suppose Alice writes and releases a free software package, and then
+    Bob creates a Debian package of that software package.  Alice is the
+    *upstream maintainer* (sometimes abbreviated as *upstream*) of the
+    package, Alice's releases are the *upstream releases*, and the version
+    number she puts on a release is the *upstream version*.  Bob may make
+    Debian-specific modifications to the package, and then later send
+    those modifications *upstream* to be incorporated in Alice's releases.
+
+    The packager and upstream developer may be the same person.  For
+    example, Alice may choose to package her own software for Debian.
+    However, this manual still distinguishes between the role of upstream
+    and the role of Debian packager, even when the same person is filling
+    both of those roles, since they have some implications for the details
+    of packaging.
+
 UTF-8
     The transformation format (sometimes called encoding) of
     `Unicode <http://www.unicode.org/>`_ defined by `RFC
diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index edae8c1..d6fbfc7 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -1,6 +1,37 @@
+.. _s-source-packages:
+
 Source packages
 ===============
 
+A Debian source package contains the source material used to construct one
+or more :doc:`binary packages <ch-binary>`.  A source package consists of
+a ``.dsc`` file (see :ref:`s-debiansourcecontrolfiles`), one or more
+compressed tar files, and possibly other files depending on the type and
+format of source package.  Binary packages are contructed from the source
+package via a build process defined by ``debian/rules`` and other files in
+the ``debian`` directory of the unpacked source package.
+
+Debian source packages are classified as *native* or *non-native*.
+
+A native source package is one that does not distinguish between Debian
+packaging and upstream releases.  A native source package contains a
+single tar file of source material, and the versioning does not have a
+Debian-specific component.  Native packages are normally (but not
+exclusively) used for software that has no independent existence outside
+of Debian, such as software written specifically to be a Debian package.
+
+A non-native source package separates the upstream release from the Debian
+packaging and any Debian-specific changes.  The source in a non-native
+source package is divided into one or more upstream tar files plus a
+collection of Debian-specific files.  (Depending on the format of the
+source package, those Debian-specific files may come in the form of
+another tar file or in the form of a compressed diff.)  The version of a
+non-native package has an upstream component and a Debian component, and
+there may be multiple Debian package versions associated with a single
+upstream release version and sharing the same upstream source tar files.
+
+Most source packages in Debian are non-native.
+
 .. _s-standardsversion:
 
 Standards conformance