Re: dh_python and python policy analysis
- To: debian-devel@lists.debian.org
- Cc: debian-python@lists.debian.org
- Subject: Re: dh_python and python policy analysis
- From: Manoj Srivastava <srivasta@debian.org>
- Date: Tue, 05 Sep 2006 16:35:16 -0500
- Message-id: <[🔎] 87k64ij8tn.fsf@glaurung.internal.golden-gryphon.com>
- In-reply-to: <87ac6eyf8y.fsf@glaurung.internal.golden-gryphon.com> (Manoj Srivastava's message of "Tue\, 08 Aug 2006 18\:33\:33 -0500")
- References: <874px1at6j.fsf@glaurung.internal.golden-gryphon.com> <87oduwxrgq.fsf@glaurung.internal.golden-gryphon.com> <200608080018.18837.madcoder@debian.org> <200608081341.22746.madcoder@debian.org> <87ac6eyf8y.fsf@glaurung.internal.golden-gryphon.com>
Hi,
This is version 1.05 of the draft, now with typo fixes, and
some initial policy support for partial upgrades for pure python
public modules that are trying to drop support for older versions of
python. The idea is that error cases are minimized if we do not drop
a version of python in a new stable that was the default version in
oldstable; at the same time insisting that every pre python public
module that says "all" in the versions supported field must support
all shipping versions of python in stable (even if this takes extra
effort in the form of splitting off private modules and including them
conditionally based on python version from a wrapper module that does
little but conditional imports.
The current version, and future updates, are to be
found at http://www.golden-gryphon.com/software/manoj-policy/
manoj
Packaging with the new Python policy
A package developers view
Manoj Srivastava
Developer
The Debian Project
Copyright (c) 2006 Manoj Srivastava
Revision History
Revision 1.0.5 4^th November 2006
Revision 1.0.4 12 Aug 2006
Revision 1.0.3 10 Aug 2006
Revision 1.0.2 8 Aug 2006
Revision 1.0.1 07 Aug 2006
Revision 1.0.0 31 Jul 2006
Specification of the new Python policy. This article grew as an attempt to
correct a gap in the concrete specification of the new Python policy, and
has grown to be close to a formal specification of the proposed new
policy.
--------------------------------------------------------------------------
Table of Contents
1. [1]Introduction
1.1. [2]Acknowledgements
2. [3]Goals of the new Python policy
3. [4]Nomenclature and definitions
3.1. [5]Python versions
3.1.1. [6]The Default Python version
3.2. [7]Categorization of Python software
4. [8]General Notes
4.1. [9]Naming python module packages
4.2. [10]Python versions supported by the source
4.3. [11]Byte compilation
4.4. [12]Linking extention modules
4.5. [13]Depends:
4.6. [14]Provides
4.7. [15]Build-Depends:
4.8. [16]Deprecating "current" in versions supported
5. [17]Recipe for developers
5.1. [18]Script
5.1.1. [19]Supported versions
5.2. [20]Private Pure Python Modules
5.2.1. [21]Byte compilation
5.2.2. [22]Supported versions
5.3. [23]Private Extension
5.3.1. [24]Supported versions
5.4. [25]Public pure Python Module
5.4.1. [26]Byte compiling
5.4.2. [27]Supported versions
5.4.3. [28]Depends:
5.5. [29]Public Extension
5.5.1. [30]Supported versions
5.5.2. [31]Provides
6. [32]Changing the default Python version
6.1. [33]Python rtupdate scripts
6.1.1. [34]rtupdate script invocation
1. Introduction
While trying to update SELinux packages, I ran across problems in trying
to determine if my packages were complying with the new python policy: any
practical tips for packaging generally devolved to the statement "Oh, just
run dh_python". This is my attempt to offer more concrete tips for
packaging. While this document started by reverse engineering dh_python,
it has, thanks to help from various people more knowledgeable about Python
than I, moved beyond that, and is closer to being a specification
unfettered by the idiosyncrasies of current tools and implementations.
So now this document is a draft formal specification of the proposed new
Python packaging policy. While it draws upon earlier documents, notable
[35]Debian Python Policy , and the [36]new policy Wiki, the [37]Debian
Python FAQ, and the source code for dh_python, and debhelper scripts, it
has essentially been written from scratch, with material reworded,
reorganized, and rearranged, to the extent that it bears little
resemblance to the original sources.
Compiled Python modules are very dependent on the specific Python
version they were compiled against, and may otherwise have restrictions on
the versions of Python they are compatible with. Unless care is taken,
introducing, or dropping, new versions of Python, or changing the default
version, trigger long and often painful transitions where the archive is
inconsistent, and the systems is ill-integrated for the duration. This new
Python policy seeks to address these potential messy transitions, and
minimize the pain.
--------------------------------------------------------------------------
1.1. Acknowledgements
While this document draws upon the expertise of multiple people and
various documents, it has benefited especially from the patience and
gentle corrections of people on the Debian-devel mailing list, and
specifically from Josselin Mouette, Loíc Minier, Pierre Habouzit, and
Matthias Klose.
--------------------------------------------------------------------------
2. Goals of the new Python policy
The new policy is designed to reduce the load on people packaging Python
[38]modules, and, by the dint of doing so, ease the transition that occur
as new Python versions are introduced, old ones removed, and as the
default version of Python changes, with minimal impact on the target
system. As far as possible, Python version transition events are handled
by automated processes, and minimal effort recompilation of new versions
when the process can not be fully automated.
* The need to support more than one version of a python runtime or to
support different implementations was seen. It takes a while for
applications to support new versions of Python, and supporting
multiple versions of Python is essential for a smooth transition
preiod.
* The old schema of using pythonX.Y-foo packages lands packages in the
NEW queue, when support for another python runtime is added to the
package. Since this requires manual intervention, support for new
versions of Python added manual processing and often delayed the
support for the new Version of python.
Having pythonX.Y-foo mentioned in the control file would disallow
binary NMU's in situations where a python runtime is dropped or added,
since the control needs to be regenerated.
* Putting extension modules for more than one python version into a
package eases transition of these packages to the testing
distribution, provided that the package supports to default python
version in testing and the default python version in unstable. [39][1]
* Ease the manual intervention required when any of the following
events occur:
New python version introduced
* Most [40]pure Python modules with no restrictions
on the version of Python supported, and those pure
Python modules that only have a lower bound on the
versions of python supported (for example, ">= 2.3",
or "all"), would require no upload; they are merely
recompiled using the rtupdate hooks, or utilities
like python-central and python-support that hook
themselves into rtupdate.
* A number of [41]public [42]extension modules that
do not have restrictions on the version of Python
supported can just be recompiled. These include
packages like:
* Packages using $(shell pyversions -s) to
determine which Python versions to build for at
build time, and which build depend upon
python-all-dev would work seamlessly.
* Some packages using CDBS will also work out
of the box.
* Packages using the distutils> build system
should also work.
* [43]Private modules are only built for one Python
version, usually the default (pyversions -d).
Private modules that do not have restrictions on the
version of Python required, or which intelligently
use $(shell pyversions -s) and their internal
restrictions to discover which version of Python to
build for would also merely need to be recompiled.
The only packages that will need manual upgrades, are
packages that needed a set of python versions that did
not contained the (at the upload time) current python
version, and a couple of other cases, which should reduce
the packages that need a non-automatized action from the
maintainer (like a new source upload). This reduction of
manual intervention should reduce the effort and time
taken for changing the Python version or adding or
dropping versions of Python much more painless than the
old policy.
Changing the [44]default Python version
The new policy aims to reduce the pressure on packagers
when the default Python version (what /usr/bin/python
points to) changes. In that case, it tries to:
* Any packages that do not need to change need not
be rebuilt or uploaded. Pure Python modules, either
public or private, should be byte compiled
automatically on the target machine using the
rtupdate mechanisms.
* Packages that have already an extension installed
for the new version of Python do not need to change
either, since they would continue to keep working.
Dropping old Python versions
The new policy also aims to reduce the pressure when an
old version of Python is dropped, since only packages
directly dependent on that version will need a manual
upgrade (this includes, for example, any package with
scripts that use /usr/bin/pythonX.Y).
The new policy also reduces the numbers of packages in the archive, by
supporting multiple versions of Python in the same binary package (at the
cost of increased size of that one package, but it should still result in
space saving.) [45][2]
This presupposes that the package build mechanism utilizes the utility
pyversions to get information about the supported Python versions, the
currently installed Python executable versions, and the default Python
runtime, instead of hard coding values in the debian/rules script. This
script can also be used to parse the value of the XS-Python-Version field
in the debian/control or debian/pyversions.
Another consequence of the current design: the default python version
has to be installed, other supported versions can be installed
additionally, not as a replacement.
--------------------------------------------------------------------------
3. Nomenclature and definitions
This section sets the scene for the rest of the document, with
definitions and glossary of terms used later in the manual.
--------------------------------------------------------------------------
3.1. Python versions
A number of different versions, or releases, of Python are available for
a Debian system. These include legacy releases, current releases, and beta
or future releases of Python. The list of currently supported versions of
Python can be found in the file /usr/share/python/debian_defaults. The
file is in plain text, divided into sections with headings in square
brackets. In each section there are lists of variables and values, one per
line, in the format variable = value. The variable default-version in the
section "DEFAULT" gives the current default Python package name, and the
variable supported-versions contains a comma separated list of package
names corresponding to the supported versions of Python in the
distribution. In order to facilitate partial upgrades, versions of Pythion
that were the default in the previous stable release of Debian are usually
retained in the current release.
In this manual, we consider the term versions to be "feature" releases
-- consisting of a major and a minor version (sub minor versions are not
distinguished).
--------------------------------------------------------------------------
3.1.1. The Default Python version
At any given time, one version of Python supported by the Debian
distribution is designated as the "default" version. The package python
will contain the current default version. [46][3] All other versions of
Python available in Debian would be called pythonX.Y
The python must provide a symbolic link from the executable
/usr/bin/python to the appropriate binary file /usr/bin/pythonX.Y.
--------------------------------------------------------------------------
3.2. Categorization of Python software
Program/script
This consists of software directly called by an end user of
external program, and is independently interpreted by the Python
interpreter. Usually starts with the magic bytes "#!", with the
interpreter being /usr/bin/python* or /usr/bin/env python*.
Modules
This is code included in python "programs/scripts", and not
invoked directly (serving as library modules in compiled
languages).
Modules can be categorized under two orthogonal criteria: firstly, based
on the whether or not they are implemented purely in python, like so:
Pure Python Module
These are python source code, to be interpreted by the Python
interpreter just like program/script code is, and may work across
many versions of Python.
Extension Module
Extensions are C code compiled for a specific version of Python,
and so may only be used by one version of Python.
Another way of categorizing modules is based on whether or not they are
available for use by third party scripts/modules.
Public
Public modules are available for use in other Python scripts or
modules using the import directive. They are installed in one of
the directories:
/usr/lib/pythonX.Y
This directory is reserved for official python
modules. No other package apart from upstream
official Python modules should install modules in
this directory.
/usr/lib/pythonX.Y/site-packages
This is where most add-on modules live. Often,
packages do not directly install modules here, but
instead use utility packages like python-central and
python-support to byte compile and install modules as
needed.
/var/lib/python-support/pythonX.Y
Packages using python-support actually have their
packages linked in from this directory, but no
package should directly install modules there
directly. See the documentation for python-support
for details.
Packages may install public Python modules in directories
specific to Python packaging utilities -- which specify the
directories under which such modules should be dropped, and the
the structure of these directories is defined by the utilities
themselves. Please note that these directories are not in the path
for Python, and are not available for modules to be imported from.
At the time of writing, such utility specific directories include:
/usr/share/pycentral
/usr/share/python-support
.
Private
Private modules are modules that are used internally by a
program, and are generally only accessible to that specific
program or suite of programs included in the same package. They
are not intended to be shared with third party programs. They are
installed in special directories, for example:
/usr/lib/<package>
/usr/share/<package>
/usr/lib/games/<package>
/usr/share/games/<package>
--------------------------------------------------------------------------
4. General Notes
One of the major features of Python packaging on Debian systems is that
pure Python modules are byte compiled for all officially supported
versions of Python that are available, and are actually supported by the
module itself. There are several utilities available to aid in this task.
Two of the most popular (and mutually exclusive -- select one or the other
for use in your packages) (python-central and python-support, for
example). However, these utilities need information about modules in the
package (for example, whether a pure Python module supports all versions
of Python or not).
Python packages (unrelated to Debian packages) are related parts of a
software system, that consist of a file __init__.py, pure Python modules,
extension modules, and other packages. These Python packages must be
packaged into the same directory structure employed by the upstream Python
package, since splitting the package components across directories, or
otherwise changing the layout, changes the layout order and may cause
breakages, subtle bugs, render the documentation inaccurate, and confuse
IDEs and other tools.
--------------------------------------------------------------------------
4.1. Naming python module packages
Packages containing public modules should be named, or should provide,
python-foo, where foo is the name of the Python module.
--------------------------------------------------------------------------
4.2. Python versions supported by the source
The XS-Python-Version field in debian/control specifies the versions of
Python supported by the package [47][4]. While this is a requirement only
if using the utility package python-central (python-support, for example,
prefers debian/pyversions), setting this is "appreciated" in any case,
according to the [48]new policy wiki[49][5]. This is used to track
packages during Python transitions, and test rebuilds.
This can be a single version, or one or more of a list of
non-overlapping ranges. The lowest range may optionally omit a low end,
and the highest range may optionally omit an upper end. In other words,
the overall range may be open ended. The ranges are often matched to the
set of all known Python version that have existed, and the supported set
is the intersection of the known versions of python and the range
specification.
1. If the current version of Python is supported by the package, then:
* For packages with private modules or private extensions
compiled for the current python version and for applications
using /usr/bin/python, this should be set to the string "all" (or
"-", in the case of debian/pyversions). [50][6] If the module
doesn't work with all Python version, the set of versions
supported should be used [51][7]
* For packages with public modules, this should be set to the
string "all" (or "-", in the case of debian/pyversions), unless
not all versions of Python are supported (in which case the
setting should specify the versions or set of versions actually
supported, like ">= 2.4" or ">= 2.2, << 2.y".
2. If the current version is not supported, or if the application uses
/usr/bin/pythonX.Y explicitly, then this should be set to the (list
of) version(s) supported.
--------------------------------------------------------------------------
4.3. Byte compilation
Any pure Python modules shipped in a package should not ship with the
corresponding byte compiled versions, since the byte compiled file is
specific to the version of Python that did the compilation. In order to
ease transitions, it is desirable that the byte compilation be done on the
target system on installation, and it should be feasible to redo the byte
compilation at will at any time (for example, when a new version of Python
is installed, or when the default version of Python changes).
So the package maintainer scripts arrange to have the pure Python
modules compiled on installation, and removed when the package is removed,
and also arrange to hook into the [52]rtupdate mechanism to redo the
byte-compilation whenever the status of the installed Python versions on
the system changes. There are a variety of tools (python-central and
python-support) to facilitate this task.
A package should only compile pure Python modules that belong to it, and
should honor the setting in the configuration file
/etc/python/debian_config which allows the local administrator to
configure how modules are byte-compiled.
--------------------------------------------------------------------------
4.4. Linking extention modules
There should be no reason to link the extension against the python
library. Usually many extensions which are developed upstream on Windows
do link by default to libpython. Other extensions linking against
libpython are those with build infrastructure maybe predating distutils.
--------------------------------------------------------------------------
4.5. Depends:
The rules for calculating the dependencies a package has are simple.
1. If a script invokes /usr/bin/pythonX.Y, than the package must depend
on pythonX.Y. This is because no amount of automatic byte compiling
would ever get rid of the requirement that /usr/bin/pythonX.Y has to
be present for the script to function.
2. For package that contains extensions, the set of Python versions
required has to be restricted to Python versions for which the
extensions have been built and shipped in the package (this is in
intersection with the set of versions restrictions already in place,
for example if the module explicitly requires some version of python).
For packages with private extension modules, this means that the set
of python versions it depends on has to be set to whatever version of
Python was used during the build process (since private extension
module packages can only be compiled for one version of Python at any
time).
3.
a.
i. If any version of python can be supported, there are two
[53][8] sub cases:
A. For pure Python modules, the package just depends on
python. This includes pure Python private modules (as
long as they do not depend on any private extensions --
in which case the extensions shall restrict the
dependency).
B. Private extension modules can only be compiled for one
version of Python at a time, so even if they have no
restrictions on the versions of python supported, they
must depend on just the current version (which is the
version they are likely to be compiled for).
To do this, one determines the stop version: the stop
version is next minor version after the current version
(found by incrementing the minor version), whether or
not it exists, and is form of Major.Minor.
In this case, the package must depend on: python (>=
$current_version), python (<< $stop_version)
b. If there is a set of values of versions of Python supported,
there are two sub cases:
i. The current version is supported. In this case, select the
larger interval in the set of Python versions supported that
contains the current version, and determine the lower and
upper bounds.
If the lower bound to the set exists, then add a dependency
on python (>= $min_version)
If the upper bound to the set exists, then the stop version
is the next minor version after the upper bound (found by
incrementing the minor version), whether or not it exists,
and is form of Major.Minor. If the upper bound to the set
exists, then depend on python (<< $stop_version)
ii. The package does not support the current version of
Python, again, there are again two sub cases:
A. The set of Python versions supported is higher than
the current version. In this case, the package should
depend upon: python (>= $min_version) |
python$min_version)
If the upper bound to the set exists, then the stop
version is the next minor version after the upper bound
(found by incrementing the minor version), whether or
not it exists, and is form of Major.Minor. If the upper
bound to the set exists, then depend on python (<<
$stop_version)
B. The set of Python versions supported is lower than
the current version. This implies that the package is
obsolete, and should either be updated to support the
current Python version, or be dropped from the archive.
Packaged modules that require other modules to work, must depend on the
corresponding python-foo packages. They must not depend on any
pythonX.Y-foo package directly.
Packaged modules available for only one particular version of Python
that need other modules (say, "bar"), must depend on the corresponding
pythonX.Y-bar packages, and must not depend on any python-bar. For
consistency, if the package ("foo") provides several pythonX.Y-foo
packages, and it needs the module "bar", it must also depend on
pythonX.Y-bar corresponding to each version "X.Y" for the virtual packages
pythonX.Y-foo that it provides.
--------------------------------------------------------------------------
4.6. Provides
Packages with public modules and extensions should be named, or should
provide, python-foo. Pure Python public modules that support all Python
versions need not have a Provides field.
For package that contains public extensions, the set of versions
supported has to be restricted to Python versions for which the extensions
have been built and shipped in the package.
Public pure Python modules that have a subset of all python versions
supported, or for public extensions, the Provides field indicates which
versions of Python are supported (for which one may import the module).
For every version of python supported the package should provide
pythonX.Y-foo packages. This assumes that the package correctly depends on
all the appropriate versions of any version specific module that it itself
requires.
--------------------------------------------------------------------------
4.7. Build-Depends:
If the package provides public extension modules, then build depending
on "python-all-dev (>= 2.3.5-11)" shall ensure that all the >pythonX.Y-dev
packages are available during building. Pure python modules do not require
any development packages at build time, since they are byte compiled on
the target machine at install time.
On the other hand, if the package provides private modules, then build
depending on "python-dev", or one specific "pythonX.Y-dev" (if there are
restrictions on the version of Python supported) is sufficient, since only
one version of Python is ever supported in any package.
If you are using helper packages (CDBS, yada, debhelper, etc) you should
build depend on the appropriate version. This also applies if your build
process uses either python-central or python-support.
--------------------------------------------------------------------------
4.8. Deprecating "current" in versions supported
Currently, the string "current" in the field XS-Python-Version is used
by python-central to indicate that the package contains private modules,
and explicitly state that the package is only built for the current Python
version, and not for any other version supported in Debian. This is
flawed, for the following reasons:
* The value corresponds to the version of Python the package is
currently built for; but in all other cases the semantics of the
XS-Python-Version field is to indicate which versions of Python are
supported by the package, and indicates compatibility, not the version
it is currently built against. So this special case breaks the
semantics of the field.
* The information conveyed by this field is redundant; it should be
clear that the package contains private modules, based on the
directories the modules are shipped in, and also the fact that it
ought to build depend on python-dev and not on python-all-dev.
* The semantics of "current" are not fixed, since they depend on the
contents of the package python-defaults, and are ill suited for the
debian/control file.
--------------------------------------------------------------------------
5. Recipe for developers
5.1. Script
These are executable scripts which start with the magic string "#!".
--------------------------------------------------------------------------
5.1.1. Supported versions
If a script invokes /usr/bin/pythonX.Y, then the version supported by
the source package (XS-Python-Version or debian/pyversions) should be
restricted to X.Y, assuming that the field is being provided. Or else, it
should be set to the list of python versions that the script can support,
or "all". [54][9] If there are separate scripts that invoke different
versions of Python, then all these versions must be in the Depends field
-- if you still want to continue packaging instead of just shooting the
upstream.
No script must use /usr/bin/python if it needs a Python version strictly
greater or strictly lower than the current version.
--------------------------------------------------------------------------
5.2. Private Pure Python Modules
These are python source code files that live in a private directory. If
a package contains a private pure Python module, only one Python version
may be supported, even if the private module can be recompiled for any new
version of Python.
--------------------------------------------------------------------------
5.2.1. Byte compilation
Any directory in which private pure Python modules are placed should be
remembered, and the modules byte-compiled on installation. This is done
automatically if one uses python-central, python-support uses the
information in the file /usr/share/python-support/$package.dirs for a list
of directories to scan.
--------------------------------------------------------------------------
5.2.2. Supported versions
The version supported by the source package (XS-Python-Version or
debian/pyversions) is either the specific version of Python supported, or
"all" [55][10] , if there are no specific restrictions based on Python
version.
This should be set to "all" [56][11] if there are no specific
restrictions based on Python version. Note that presence of private
extension modules in the same package would mean that there are
restrictions on the XB-Python-Version field.
If only a subset of Python versions are supported by the package, then
the list of versions supported should be used [57][12]
--------------------------------------------------------------------------
5.3. Private Extension
These are compiled files linked to python libraries, and kept in a
private directory. Since these files are compiled with one specific
version of python, and do not live in versioned directories, only one
version of python is supported at any given time.
--------------------------------------------------------------------------
5.3.1. Supported versions
The version supported by the source package (XS-Python-Version or
debian/pyversions) is either the specific version of Python supported, or
"all" [58][13], if there are no specific restrictions based on Python
version.
If a single version of Python is supported, then the version supported
by the binary package (XB-Python-Version field or the file .versions) is
set to that version (copied from XS-Python-Version). If the current
default version of Python is not supported, this field it set to the
minimum version of Python actually supported by the module. If the current
default version of Python is supported (or there are no restrictions on
the version of python supported), then this field is set to the current
version.
--------------------------------------------------------------------------
5.4. Public pure Python Module
Public modules should be packaged with a name of python-foo, where foo
is the name of the module. Such a package should support the current
Debian Python version, and more if possible.
There are two kinds of public pure Python modules, the most common being
the variety that live in unversioned public module directories, and, in
rare cases, pure python modules that live in versioned public module
directories. The latter is usually the case when the pure Python module
imports a public extension module from the same directory, and thus the
public extension and pure python modules must be in the same directory.
Otherwise, pure python modules should live in an unversioned public module
directory. Pure python versions that live in unversioned directories must
support all versions of Python being shipped by Debian[59][14]
Depending on the packaging utility used, the modules live in either
/usr/share/python-central or in /usr/share/python-support/$package.
Pure Python modules belonging to the core Python package generally live
in a different set of directories than unofficial ones, but are otherwise
treated exactly like other public pure Python Module which live in
unversioned directories as detailed below.
--------------------------------------------------------------------------
5.4.1. Byte compiling
In the common case of pure Python modules in unversioned public module
directories, tools exist to help byte compile the pure Python modules for
all versions of Python installed on the target system. In case of pure
Python modules in versioned public module directories, byte compilation is
up to the package scripts.
--------------------------------------------------------------------------
5.4.2. Supported versions
For the source package, this should be the version of Python supported,
or "all", if there are no specific restrictions based on Python version.
In case of pure Python modules in versioned public module directories, the
set of versions supported usually corresponds to the list of versioned
directories in which pure Python modules live.
--------------------------------------------------------------------------
5.4.3. Depends:
Dependencies for the package also differ slightly based on whether the
pure Python module is version dependent or not.
1. In the common case of pure Python modules in unversioned public
module directories,
a. If there is a minimum supported version,
i. If the minimum version supported is less than or equal to
the current version, then depend on python (>=
$min_version)
ii. Or else, if the minimum version supported is greater than
the current version, then depend on python (>=
$min_version) | python$min_version
If a upper bound of the supported versions exists, then the
stop version is the next minor version after the maximum
supported version (found by incrementing the minor version),
whether or not it exists, and is form of Major.Minor. If there is
an upper bound on the supported versions, and the upper bound is
greater than or equal to the current version, then depend on
python (<< $stop_version)
b. If the current version is higher than any supported version,
then this package is obsolete. However, the following
dependencies are still valid, if the corresponding lower and
upper bounds of supported versions exist. python (>=
$min_version) | python$min_version, python (<< $stop_version)
2. In case of pure Python modules in versioned public module
directories, the set of versions corresponding to the list of
versioned public module directories in which pure Python modules live
is used in lieu of the value of XS-Python-Version.
a. If the lowest version in the set is less than or equal to the
current version, then depend on python (>= $min_version)
b. Or else depend on python (>= $min_version) |
python$min_version,
c. If the highest version in the set is greater than or equal to
the current version, then depend on python (<< $stop_version)
where the stop version is the next minor version greater than the
highest supported version.
d. For every version in the set, also depend on python$version
--------------------------------------------------------------------------
5.5. Public Extension
Public extensions should be packaged with a name of python-foo, where
foo is the name of the module. Such a package should support the current
Debian Python version, and more if possible.
Depending on the packaging utility used, the modules live in either
/usr/lib/pythonX.Y/site-packages or in
/usr/lib/python-support/$package/pythonX.Y, with proper symbolic links in
place.
--------------------------------------------------------------------------
5.5.1. Supported versions
The versions supported by the source and binary packages should be set
to the list of versions for which the compiled extension modules are
provided.
--------------------------------------------------------------------------
5.5.2. Provides
For every version of Python for which compiled extension modules are
provided, provide a package named pythonX.Y-foo (assuming that the package
name is python-foo).
--------------------------------------------------------------------------
6. Changing the default Python version
Some packages may need more changes during an update of the default
python version (i.e. setting an alternative, restarting a daemon). These
packages can either provide their own upgrade scripts (run during the
update of the default python version), or continue to use the old
dependency on exactly the current python version to ensure that the
package is updated together with the default python version and use the
normal installation/removal scripts. The utility packages python-central
and python-support provide upgrade scripts in order to facilitate their
byte compilation work. [60][15]
--------------------------------------------------------------------------
6.1. Python rtupdate scripts
When the default python version is changed, the maintainer scripts of
the python package arrange for the execution of package provided scripts.
These scripts are placed in /usr/share/python/runtime.d/, must have an
.rtupdate extension and must be executable. These scripts must be
idempotent; since any script may be called multiple times during a Python
version change with the same arguments.
These scripts must be rigorously tested, since any failure in the
scripts would make the new Python package changing the default version to
fail to install.
Each script is called with one of pre-rtupdate, rtupdate or
post-rtupdate as the first parameter. If nothing needs to be done the
invocation should be ignored by the script.
--------------------------------------------------------------------------
6.1.1. rtupdate script invocation
1. in the pre-installation phase of the python package, the package
supplied scripts are called with the parameter: pre-rtupdate <old
runtime> <new runtime>
A failure in any script results in the failure of the
pre-installation script of the python package.
[61]Note Whether or not all scripts are run, or the process aborts
at the first failure, is still under flux
At this point, the failed-pre-rtupdate hook is run, to allow the
package to go to a sane state again. Otherwise, such a failure of a
pre-rtupdate script would leave all packages whose pre-rtupdate has
been run in a dangling state, making a bug in a pre-rtupdate a
critical bug. Still, be very very careful when working on a
pre-rtupdate script.
2. During the post installation phase of the python package, the
package supplied scripts are called with the parameter: rtupdate <old
runtime> <new runtime>
This is the hook used by python-central and python-support to byte
compile pure Python modules; and packages handling their own byte
compilation should use this hook as well. However, at this phase of
the installation process, the state of the default Python package is
in flux, so complex operations involving the default version are best
avoided.
A failure in any script results in the failure of the
post-installation script of the python package, and may leave other
Python packages in an non-working state (since the modules may not be
compiled with the default version). A failure in this script is also a
critical bug.
3. During the post installation phase of the python package, the
package supplied scripts are called with the parameter: post-rtupdate
<old runtime> <new runtime>
A failure in any script results in the failure of the
post-installation script of the python package. Such a failure only
affects the package which owns the script, and does not impact the
other related packages, so a problem in the post-rtupdate script is
less critical, and would only have the severity grave or serious.
Notes
[62][1] The older policy required an extra upload of every package
containing an extension, adding new dependencies on new shared
libraries in unstable, but not yet in testing. It also tended to
require all packages that required a version of python strictly
less than a specific version to move into testing at the same
time, creating long periods where packages were blocked.
[63][2] The two cases where this happens is
* Public extension modules, where extensions compiled for
multiple versions of Python are shipped in the same package,
and
* Pure modules where different versions of the upstream
package are shipped and are directly installed into
/usr/lib/pythonX.Y.
[64][3] Usually, the default Debian Python version is the latest stable
upstream release that can be integrated in the distribution.
[65][4] python-central also uses XB-Python-Version field in the control
file, which is substituted in the binary package using the values
from the XS-Python-Version field, filtered according to various
rules.
[66][5] For compatibility, each of the tools (python-central and
python-support) are able to use each others data.
[67][6] Note that pycentral uses the special string "current" here.
[68][7] For pycentral, one may specify something like "current, >= 2.4"
or even "current, >= 2.2, << 2.5". Packages with private
extention modules also can use the value "current_ext", which
will will expand to "current", if the package does not have any
extensions and can be used with another python default version
without a new upload. It's replaced by the version number of the
current default version in the Python:Versions substitution
variable.
[69][8] Public extensions are restricted to the list of Python versions
for which the extensions have been compiled, and thus are covered
in the section below.
[70][9] If the package uses python-central, and if there is no other
value for the versions (set by any modules packaged), then the
versions supported by the binary package (XB-Python-Version) is
the same as above (XS-Python-Version, if XS-Python-Version is not
empty, or "all", if it is empty).
[71][10] If using python-central, use the special string "current"
instead of "all".
[72][11] If using python-central, use the special string "current"
instead of "all". Note that if the package contains private
extension modules, then the rules for setting XB-Python-Version
for the private extension modules take precedence.
[73][12] python-central differs here. If the current version is
supported, then XB-Python-Version should be set to "current".
If the current version is not supported, and if the minimum
version supported is strictly higher than the current version,
the XB-Python-Version field should be set to the minimum version
supported.
Failing that, if the current version is not supported, and the
max version is lower than the current version, then this package
should be deprecated, and is obsolete.
[74][13] If using python-central, use the special string "current"
instead of "all".
[75][14] This might require the maintainer to take extraordinary measuer
if upstream drops support for version of python that Debian
ships, or starts using features found in new versions of Python
alone. In order to support partial upgrades from oldStable to
Stable, or from Stable to testing, the version of Python that was
the default in the older release must still be supported -- even
if it requires shipping the real code in private modules, and
conditionally importing old or new variants based on the version
of Python.
[76][15] This section is based on [77]basic rtupdate documentation
References
Visible links
1. file:///tmp/html-i20017#AEN96
2. file:///tmp/html-i20017#AEN109
3. file:///tmp/html-i20017#AEN124
4. file:///tmp/html-i20017#AEN221
5. file:///tmp/html-i20017#AEN224
6. file:///tmp/html-i20017#AEN235
7. file:///tmp/html-i20017#AEN249
8. file:///tmp/html-i20017#AEN326
9. file:///tmp/html-i20017#AEN333
10. file:///tmp/html-i20017#AEN338
11. file:///tmp/html-i20017#AEN392
12. file:///tmp/html-i20017#AEN401
13. file:///tmp/html-i20017#AEN404
14. file:///tmp/html-i20017#AEN479
15. file:///tmp/html-i20017#AEN489
16. file:///tmp/html-i20017#AEN503
17. file:///tmp/html-i20017#AEN523
18. file:///tmp/html-i20017#AEN525
19. file:///tmp/html-i20017#AEN529
20. file:///tmp/html-i20017#AEN548
21. file:///tmp/html-i20017#AEN551
22. file:///tmp/html-i20017#AEN557
23. file:///tmp/html-i20017#AEN586
24. file:///tmp/html-i20017#AEN589
25. file:///tmp/html-i20017#AEN604
26. file:///tmp/html-i20017#AEN614
27. file:///tmp/html-i20017#AEN617
28. file:///tmp/html-i20017#AEN621
29. file:///tmp/html-i20017#AEN671
30. file:///tmp/html-i20017#AEN677
31. file:///tmp/html-i20017#AEN680
32. file:///tmp/html-i20017#AEN685
33. file:///tmp/html-i20017#AEN693
34. file:///tmp/html-i20017#AEN704
35. http://www.debian.org/doc/packaging-manuals/python-policy/
36. http://wiki.debian.org/DebianPython/NewPolicy
37. http://wiki.debian.org/DebianPythonFAQ
38. file:///tmp/html-i20017#mod001
39. file:///tmp/html-i20017#FTN.AEN139
40. file:///tmp/html-i20017#pure001
41. file:///tmp/html-i20017#pub001
42. file:///tmp/html-i20017#ext001
43. file:///tmp/html-i20017#pri001
44. file:///tmp/html-i20017#AEN235
45. file:///tmp/html-i20017#FTN.AEN203
46. file:///tmp/html-i20017#FTN.AEN240
47. file:///tmp/html-i20017#FTN.AEN343
48. http://wiki.debian.org/DebianPython/NewPolicy
49. file:///tmp/html-i20017#FTN.AEN353
50. file:///tmp/html-i20017#FTN.AEN368
51. file:///tmp/html-i20017#FTN.AEN372
52. file:///tmp/html-i20017#AEN693
53. file:///tmp/html-i20017#FTN.AEN426
54. file:///tmp/html-i20017#FTN.AEN538
55. file:///tmp/html-i20017#FTN.AEN563
56. file:///tmp/html-i20017#FTN.AEN570
57. file:///tmp/html-i20017#FTN.AEN578
58. file:///tmp/html-i20017#FTN.AEN595
59. file:///tmp/html-i20017#FTN.AEN608
60. file:///tmp/html-i20017#FTN.AEN690
62. file:///tmp/html-i20017#AEN139
63. file:///tmp/html-i20017#AEN203
64. file:///tmp/html-i20017#AEN240
65. file:///tmp/html-i20017#AEN343
66. file:///tmp/html-i20017#AEN353
67. file:///tmp/html-i20017#AEN368
68. file:///tmp/html-i20017#AEN372
69. file:///tmp/html-i20017#AEN426
70. file:///tmp/html-i20017#AEN538
71. file:///tmp/html-i20017#AEN563
72. file:///tmp/html-i20017#AEN570
73. file:///tmp/html-i20017#AEN578
74. file:///tmp/html-i20017#AEN595
75. file:///tmp/html-i20017#AEN608
76. file:///tmp/html-i20017#AEN690
77. http://people.debian.org/~doko/tmp/python-rtupdate.txt
--
A diplomat is man who always remembers a woman's birthday but never
her age. Robert Frost
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
Reply to: