Re: new dh_python proposal
> Here's updated dh_python proposal. Main change since my last mail:
> pycompile/pyclean will no longer search for files to compile/remove
> in public directories as Matthias will fix #552595 using triggers so I
> don't
> have to care about pycentral bugs anymore and dpkg -L seems to be fast
> enough
> to be used in maintainer scripts (BTW: my previous proposal had a typo:
> "prerm"
> should be replaced with "postrm", I guess everyone figured that out from
> the
> contextâ?¦).
>
> Again, the main idea is to ship files in official site/dist-packages
> location
> or symlink files from /usr/share/py{,3}shared directory there (if files
> can be
> shared).
>
> This means Architecture:all packages will need sourceful uploads once list
> of
> supported Python versions will change (binNMUs do not touch architecture
> independent packages and even if they'd do, that still leaves the
> "=${source:Version}" problem).
In line with this, I have been working on Python policy updates to
encompass Python 3 and make sure that this proposal and policy were
aligned. Attached is a draft. This is mostly my work with only some
minimal review by a few people. It is definitely a work in progress (I
ran out of time and have not verified I successfully addressed what
comments I have gotten).
I've attached both the full text of my current draft and a diff from
what's in Testing/Unstable now.
Comments please.
Scott K
Debian Python Policy
--------------------
Neil Schemenauer <nas@debian.org>
Matthias Klose <doko@debian.org>
Gregor Hoffleit <flight@debian.org>
Josselin Mouette <joss@debian.org>
Joe Wreschnig <piman@debian.org>
Loi"c Minier <lool@debian.org>
Scott Kitterman <scott@kitterman.com>
version 0.9.1.0
-------------------------------------------------------------------------------
Abstract
--------
This document describes the packaging of Python within the Debian
GNU/Linux distribution and the policy requirements for packaged Python
programs and modules.
Copyright Notice
----------------
Copyright (C) 1999, 2001, 2003, 2006, 2009 Software in the Public
Interest
This manual is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version.
This is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
A copy of the GNU General Public License is available as
`/usr/share/common-licences/GPL' in the Debian GNU/Linux distribution
or on the World Wide Web at The GNU General Public License
(http://www.gnu.org/copyleft/gpl.html).
You can also obtain it by writing to the Free Software Foundation,
Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
-------------------------------------------------------------------------------
Contents
--------
1. Python Packaging
1.1. Versions
1.2. Main packages
1.3. Minimal packages
1.4. Python Interpreter
1.4.1. Interpreter Name
1.4.2. Interpreter Location
1.5. Module Path
1.6. Hooks for updates to installed runtimes
1.7. Documentation
2. Packaged Modules
2.1. Types of Python Modules
2.2. Module Package Names
2.3. Specifying Supported Versions
2.3.1. Build time determination for python2.<X> versions
2.3.2. Build time determinations for python3.<X> versions
2.3.3. Run Time/Binary Package Versions
2.4. Dependencies
2.5. Provides
2.6. Modules Byte-Compilation
3. Python Programs
3.1. Programs using the default Python
3.1.1. Programs Shipping Private Modules
3.2. Programs Using a Particular Python Version
4. Programs Embedding Python
4.1. Building Embedded Programs
4.2. Embedded Python Dependencies
5. Interaction with Locally Installed Python Versions
A. Build Dependencies
B. Packaging Tools
B.1. distutils
B.2. python-support
B.3. python-central
B.4. CDBS
C. Upgrade Procedure
-------------------------------------------------------------------------------
1. Python Packaging
-------------------
1.1. Versions
-------------
At any given time, the binary package `python' will represent the
current default Debian Python version.
The default Debian Python version should always be the latest stable
upstream release that can be fully integrated in the distribution.
There may be newer supported or unsupported versions included in the
distribution if they are not fully integrated for a particular
release.
Apart from the default version, legacy versions of Python or beta
versions of future releases may be included as well in the
distribution, as long as they are needed by other packages, or as long
as it seems reasonable to provide them. Python3 versions are a
seperate series from Python(2) versions. Unless specifically
mentioned, the policy for Python versions also applies to Python3
versions. (Note: For the scope of this document, Python versions are
synonymous to feature releases, i.e. Python 2.5 and 2.5.1 are
sub-minor versions of the same Python version 2.5, but Python 2.4 and
2.5 are indeed different versions.)
For any version, the main binary package must be called
`python<X>.<Y>'.
The set of currently supported Python versions can be found in
`/usr/share/python/debian_defaults'. The public interface to this
file is `/usr/bin/pyversions' for Python2 versions and
`/usr/bin/py3versions' for Python3. It includes default-version,
which is the current default Python runtime, supported-versions which
is the set of runtimes currently supported and for which modules
should be built and byte-compiled, old-versions which is the list of
runtimes which might still be on the system but for which should not
be built anymore, and unsupported-versions which is the list of
runtimes which should not be supported at all, that is modules should
not be built or byte-compiled for these. This file may list both a
default Python version and a default Python3 version.
unsupported-versions is a superset of (includes) old-versions and the
default-version is always in supported-versions.
Newer versions might also appear in unsupported-versions before being
moved to supported-versions.
1.2. Main packages
------------------
For every Python version provided in the distribution, the binary
package `python<X>.<Y>' shall provide a complete distribution for
_deployment_ of Python scripts and applications. The package must
ensure that the binary `/usr/bin/python<X>.<Y>' is provided.
Installation of `python<X>.<Y>' shall provide the modules of the
upstream Python distribution with some exceptions.
Excluded are modules that cannot be included for licensing reasons
(for example the profile module), for dependency tracking purposes
(for example the GPL-licensed gdbm module) or that should not be
included for packaging reasons (for example the tk module which
depends on X.Org). Some tools and files for the _development_ of
Python modules are split off in a separate binary package
`python<X>.<Y>-dev'. Documentation will be provided separately as
well.
At any time, the `python' binary package must ensure that
`/usr/bin/python' is provided as a symlink to the current
`python<X>.<Y>' executable. `/usr/bin/python3' is to be provided
similarly by the `python3' binary package. The `python' binary
package must also depend on the appropriate `python<X>.<Y>' to ensure
this runtime is installed.
The version of the `python' binary package must be greater than or
equal to <X>.<Y> and smaller than <X>.<Y+1>.
1.3. Minimal packages
---------------------
For every Python version provided in the distribution, the binary
package `python<X>.<Y>'-minimal might exist and should not be depended
upon by other packages except the Python runtime packages themselves.
1.4. Python Interpreter
-----------------------
1.4.1. Interpreter Name
-----------------------
Python scripts depending on the default Python version (see Section
1.2, `Main packages') or not depending on a specific Python version
should use `python' (without a version) as the interpreter name.
Python scripts that only work with a specific Python version must
explicitly use the versioned interpreter name (`python<X>.<Y>').
1.4.2. Interpreter Location
---------------------------
The preferred specification for the Python interpreter is
`/usr/bin/python' or `/usr/bin/python<X>.<Y>'. This ensures that a
Debian installation of Python is used and all dependencies on
additional Python modules are met.
If a maintainer would like to provide the user with the possibility to
override the Debian Python interpreter, he may want to use
`/usr/bin/env python' or `/usr/bin/env python<X>.<Y>'. However this
is not advisable as it bypasses Debian's dependency checking and makes
the package vulnerable to incomplete local installations of Python.
1.5. Module Path
----------------
By default, Python modules are searched in the directories listed in
the PYTHONPATH environment variable and in the sys.path Python
variable. Since python2.4 version 2.4.5-3, python2.5 version 2.5.2-7,
and python2.6 version 2.6.2-1 sys.path does not include a
/usr/lib/python<X><Y>.zip entry anymore. Directories with private
Python modules must be absent from the sys.path. Public Python
modules not handled by python-central or python-support must be
installed in the system Python modules directory,
/usr/lib/python<X>.<Y>/dist-packages for python2.6 and later, and
/usr/lib/python<X>.<Y>/site-packages for python2.5 and earlier. As an
exception to the above, modules managed by python-support are
installed in another directory which is added to the sys.path using
the .pth mechanism. The .pth mechanism is documented in the Python
documentation of the `site' module. For Python3, the only public
module installation location is /usr/lib/python<X>.<Y>/dist-packages.
A special directory is dedicated to public Python modules installed by
the local administrator, /usr/local/lib/python<X>.<Y>/dist-packages
for python2.6 and later, and
/usr/local/lib/python<X>.<Y>/site-packages for python2.5 and earlier.
For a local installation by the administrator of python2.6 and later,
a special directory is reserved to Python modules which should only be
available to this Python, /usr/local/lib/python<X>.<Y>/site-packages.
Unfortunately, for python2.5 and earlier this directory is also
visible to the system Python. Additional information on appending
site-specific paths to the module search path is available in the
official documentation of the site module.
When binary packages ship identical source code for multiple Python
versions, for instance /usr/lib/python2.6/dist-packages/foo.py and
/usr/lib/python2.5/site-packages/foo.py, these should point to a
common file. A common location to share, across Python versions,
arch-independent files which would otherwise go to the directory of
system public modules is /usr/share/pyshared. For Python3, this
location is /usr/share/py3shared.
1.6. Hooks for updates to installed runtimes
--------------------------------------------
The `python' binary package has special hooks to allow other packages
to act upon updates to the installed runtimes. This mechanism is
required to handle changes of the default Python runtime in some
packages and to enable the Python packaging helpers. There are three
supported hook types which come in the form of scripts which are
invoked from the maintainer scripts of the Python runtime packages
when specific installations, removals, or upgrades occur.
1. /usr/share/python/runtime.d/*.rtinstall: these are called when a
runtime is installed or becomes supported. The first argument is
"rtinstall", the second argument is the affected runtime (for
example python<X>.<Y>) and the third and fourth argument are the
old and new version of this packaged runtime if this runtime was
already installed but unsupported.
2. /usr/share/python/runtime.d/*.rtremove: these are called when a
runtime is installed or stops being supported. The first
argument is "rtremove", and the second argument is the affected
runtime (for example python<X>.<Y>).
3. /usr/share/python/runtime.d/*.rtupate: these are called when the
default runtime changes. The first argument is either
"pre-rtupdate", called before changing the default runtime, or
"rtupdate", called when changing the default runtime, or
"post-rtupdate", called immediately afterwards. The second
argument is the old default runtime (for example python<X>.<Y>),
and the third argument is the new default runtime (for example
python<X>.<Z>).
1.7. Documentation
------------------
Python documentation is split out in separate binary packages
`python<X>.<Y>-doc'. The binary package `python-doc' will always
provide the documentation for the default Debian Python version.
TODO: Policy for documentation of third party packages.
-------------------------------------------------------------------------------
2. Packaged Modules
-------------------
The goal of these policies is to reduce the work necessary for Python
transitions. Python modules are internally very dependent on a
specific Python version. However, we want to automate recompiling
modules when possible, either during the upgrade itself
(re-byte-compiling pyc and pyo files) or shortly thereafter with
automated rebuilds (to handle C extensions). These policies encourage
automated dependency generation and loose version bounds whenever
possible.
2.1. Types of Python Modules
----------------------------
There are two kinds of Python modules, "pure" Python modules, and
extension modules. Pure Python modules are Python source code that
works across many versions of Python. Extensions are C code compiled
and linked against a specific version of the python runtime, and so
can only be used by one version of Python. Some distributions link
extensions to libpython, but this is not the case in Debian as symbols
might as well be resolved by `/usr/bin/python<X>.<Y>' which is not
linked to libpython.
Python packages are directories containing at least a `__init__.py',
other modules, extensions, or packages (A package in the Python sense
is unrelated to a Debian package). Python packages must be packaged
into the same directory (as done by upstream). Splitting components
of a package across directories changes the import order and may
confuse documentation tools and IDEs.
There are two ways to distribute Python modules. Public modules are
installed in a public directory as listed in Section 1.5, `Module
Path'. They are accessible to any program. Private modules are
installed in a private directory such as `/usr/share/<package-name>'
or `/usr/lib/<package-name>'. They are generally only accessible to a
specific program or suite of programs included in the same package.
2.2. Module Package Names
-------------------------
Public modules used by other packages must have their binary package
name prefixed with <python->. It is recommended to use this prefix
for all packages with public modules as they may be used by other
packages in the future. The binary package for module foo should
preferably be named `python-<foo>', if the module name allows, but
this is not required if the binary package ships multiple modules. In
the latter case the maintainer chooses the name of the module which
represents the package the most. For modules that support Python3,
they should be named `python3-<foo>'. Such a package should support
the current Debian Python version, and more if possible (there are
several tools to help implement this, see Appendix B, `Packaging
Tools'). For example, if Python 2.3, 2.4, and 2.5 are supported, the
Python command
import foo
should import the module when the user is running any of
`/usr/bin/python2.3', `/usr/bin/python2.4', and `/usr/bin/python2.5'.
This requirement also applies to extension modules; binaries for all
the supported Python versions should be included in a single package.
Namespace packages require some additional considerations. Only one
one package can provide a namespace (common __init__.py file) and
other packages that use the same namespace must depend on this
package. Package providing the top level namespace should be named
using the same rules as other Python binary module packages. If this
is not feasible, python-namespace-common should be used. Sub-packages
built from the same source should be named using namespace + package
separated by a period, i.e. `python-<foo>.<bar>'.
2.3. Specifying Supported Versions
----------------------------------
2.3.1. Build time determination for python2.<X> versions
--------------------------------------------------------
The optional `XS-Python-Version' field in `debian/control' specifies
the versions of Python supported by the source package. When not
specified, it defaults to all currently supported Python versions. It
is notably used to track packages during Python transitions, and is
also used by some packaging scripts to automatically generate
appropriate Depends and Provides lines. The format of the field may
be one of the following:
XS-Python-Version: >= X.Y
XS-Python-Version: >= A.B, << X.Y
XS-Python-Version: A.B, X.Y
XS-Python-Version: all
The keyword "all" means that the package supports any Python version
available but might be deprecated in the future since using version
numbers is clearer than "all" and encodes more information. The
keyword "current" has been deprecated and used to mean that the
package would only have to support a single version (even across
default version changes).
Optionally, the python3.<X> approach described below may also be used
via `/usr/bin/pyversions', but only in cases where generating
appropriate Depends and Provides lines with this approach is
supported. For pyversions, it will always return values less than
3.0.
2.3.2. Build time determinations for python3.<X> versions
---------------------------------------------------------
Packages should be built for all currently supported Python versions
for which they are compatible. The defined interface for determining
which Python versions are supported is `/usr/bin/py3versions'. The
output should be used to generate appropriate Depends and Provides
information at package build time. The format to call py3versions is
the same format used specify package version requirements in
`debian/control' (for py3versions the lowest version it will return is
3.0):
/usr/bin/py3versions ">= X.Y"
/usr/bin/py3versions ">= A.B, << X.Y"
/usr/bin/py3versions "A.B, X.Y"
The keywords "all" and "current" that were used with Python2.<X> are
not applicable to Python3.<X>.
2.3.3. Run Time/Binary Package Versions
---------------------------------------
Your control file must also, for each Python related binary, have a
line:
XB-Python-Version: ${python:Versions}
The python:Versions is substituted by the supported Python versions of
the binary package, based on the build time determination described
above. For python2.<X> packages not containing extensions the
substitution may produces any output format permitted for the
`XS-Python-Version'. For all packages with extensions and all
python3.<X> packages, the expansion must list the versions explicitly.
XB-Python-Version: 3.0, 3.1
Directly hard coding XB-Python-Version values in debian/control is
deprecated. The primary purpose of XB-Python-Version is to support
automating the detection of packages needing update when the supported
Python versions change. Hard coded values or non-specific values like
all or current do not support this design goal.
If your package is used by another module or application that requires
a specific Python version, it should also `Provide: python<X>.<Y>-foo'
for each version it supports.
2.4. Dependencies
-----------------
Packaged modules available for the default Python version (or many
versions including the default) as described in Section 2.2, `Module
Package Names' must depend on "`python (>= <X>.<Y>')". If they
require other modules to work, they must depend on the corresponding
`python-foo'. They must not depend on any `python<X>.<Y>-foo'.
Packaged modules available for one particular version of Python must
depend on the corresponding `python<X>.<Y>' package instead. If they
need other modules, they must depend on the corresponding
`python<X>.<Y>-foo' packages, and must not depend on any `python-foo'.
2.5. Provides
-------------
Provides in binary packages of the form `python-<foo>' must be
specified, if the package contains an extension for more than one
python version. Provides should also be added on request of
maintainers who depend on a non-default python version.
2.6. Modules Byte-Compilation
-----------------------------
If a binary package provides any binary-independent modules (`foo.py'
files), the corresponding byte-compiled modules (`foo.pyc' files) and
optimized modules (`foo.pyo' files) must not ship in the package.
Instead, they should be generated for all Python versions at build
time (to ensure the byte compilation will succeed on user's systems),
discarded, and then generated on the installed system in the package's
postinst. They must be removed in the package's prerm. The package's
prerm has to make sure that both `foo.pyc' and `foo.pyo' for all
Python versions are removed.
A binary package must only byte-compile the files which belong to the
package.
The file `/etc/python/debian_config' allows configuration how modules
should be byte-compiled. The postinst scripts should respect these
settings.
Pure Python modules in private installation directories that are
byte-compiled with the default Python version must be forcefully
byte-compiled again when the default Python version changes. Public
Python extensions should be bin-NMUed. Private Python extensions
should be subject to binary NMUs every time the default interpreter
changes, unless the extension is updated through a .rtupdate script.
-------------------------------------------------------------------------------
3. Python Programs
------------------
3.1. Programs using the default Python
--------------------------------------
Programs that can run with any version of Python must begin with
`#!/usr/bin/python' or `#!/usr/bin/env python' (the former is
preferred). They must also specify a dependency on `python', with a
versioned dependency if necessary.
If the program needs the Python module `foo', it must depend on the
real package providing this module, usually `python-foo' but this name
might vary when the package ships multiple modules.
3.1.1. Programs Shipping Private Modules
----------------------------------------
A program using `/usr/bin/python' as interpreter can come up with
private Python modules. These modules should be installed in
`/usr/share/<module>', or `/usr/lib/<module>' if the modules are
architecture-dependent (e.g. extensions).
The rules explained in Section 2.6, `Modules Byte-Compilation' apply
to those private modules: the byte-compiled modules must not be
shipped with the binary package, they should be generated in the
package's postinst, using the current default Python version, and
removed in the prerm. Modules should be byte-compiled using the
current default Python version.
Programs that have private compiled extensions must either handle
multiple version support themselves, or declare a tight dependency on
the current Python version (e.g. `Depends: python (>= 2.4), python
(<= 2.5)'. No tools currently exist to alleviate this situation.
3.2. Programs Using a Particular Python Version
-----------------------------------------------
A program which requires a specific version of Python must begin with
`#!/usr/bin/python<X>.<Y>' (or `#!/usr/bin/env python<X>.<Y>'). It
must also specify a dependency on `python<X>.<Y>' and on any
`python<X>.<Y>-foo' package providing necessary modules. It should
not depend on any `python-foo' package, unless it requires a specific
version of the package (since virtual packages cannot be versioned).
If this is the case, it should depend on both the virtual package and
the main package (e.g. `Depends: python2.4-foo, python-foo (>=
1.0)').
The notes on installation directories and byte-compilation for
programs that support any version of Python also apply to programs
supporting only a single Python version. Modules to be byte-compiled
should use the same Python version as the package itself.
-------------------------------------------------------------------------------
4. Programs Embedding Python
----------------------------
4.1. Building Embedded Programs
-------------------------------
Programs which embed a Python interpreter must declare a
`Build-Depends' on `python<X>.<Y>-dev', where python<X>.<Y> is the
Python version the program builds against. It should be the current
default Python version unless the program does not work correctly with
this version.
4.2. Embedded Python Dependencies
---------------------------------
Dependencies for programs linking against the shared Python library
will be automatically created by `dpkg-shlibdeps'. The
`libpython<X>.<Y>.so.<Z>' library the program is built against is
provided by the `python<X>.<Y>' package.
-------------------------------------------------------------------------------
5. Interaction with Locally Installed Python Versions
-----------------------------------------------------
As long as you don't install other versions of Python in your path,
Debian's Python versions won't be affected by a new version.
If you install a different sub-release of the version of Python you
have got installed, you will need to be careful to install all the
modules you use for that version of Python too.
-------------------------------------------------------------------------------
A. Build Dependencies
---------------------
Build dependencies for Python dependent packages must be declared for
every Python version that the package is built for. The
`python-all-dev' should be used when building modules for any or all
Python versions. To build for a specific version or versions,
Build-Depend on `python<X>.<Y>-dev'.
Some applications and pure Python modules may be able to build-depend
only on `python' or `python-all' and not require the -dev packages.
For Python3, use `python3-all-dev' or `python3-all'
Build-Depend on at least:
Build-Depends: python2.3 (>= 2.3-1)
Build-Depends: python2.4 (>= 2.4-1)
Build-Depends: python (>= 2.3.5-7)
Build-Depends: python-all
Build-Depends: python2.3-dev (>= 2.3-1)
Build-Depends: python2.4-dev (>= 2.4-1)
Build-Depends: python-dev (>= 2.3.5-7)
Build-Depends: python-all-dev
If you use either `python-support' or `python-central' you must
additionally Build-Depend on those.
-------------------------------------------------------------------------------
B. Packaging Tools
------------------
This section describes the various tools to help package Python
programs and modules for Debian. None of these tools are mandatory.
This appendix is just an overview. If you use these tools, you should
read their full documentation.
B.1. distutils
--------------
The standard Python distutils module has been modified in Debian to
change the default installation directory of public Python modules and
to add a new flag to the "install" command to override the default,
`--install-layout='. To allow the use this flag, maintainers should
ensure that at least version 2.6.2-1 will be used for python2.6,
version 2.5.4-1 for python2.5, and version 2.4.6-2 for python2.4.
This flag is parsed but ignored in python2.4 and python2.5. Public
Python modules installed with a modified distutils default to
/usr/local/lib/python<X>.<Y>/dist-packages for python2.6 and later.
This directory is seen by the system-provided python2.6. When using a
system-provided python2.4 or python2.5, the default is
/usr/lib/python<X>.<Y>/site-packages which is seen by the
system-provided python2.4 and python2.5 versions, but not by a
system-provided python2.6 and later versions. When using a local
Python installation, the default is
/usr/local/lib/python<X>.<Y>/site-packages which is only seen by the
local Python installation. Using the `--install-layout=deb' flag to
the "install" command of `setup.py' with a system-provided python2.6
or later versions, Python modules will be installed to
/usr/lib/python<X>.<Y>/dist-packages which is only seen by the
system-provided Python, not by a local installation. Using the
`--install-layout=deb' flag to `setup.py' with a system-provided
python2.4 or python2.5 does not affect the default installation
directory.
B.2. python-support
-------------------
The python-support system provides a simple way to byte-compile pure
Python modules and manage dependencies. It integrates with
`debhelper', manages byte-compilation, private modules, will properly
use the /usr/share/pyshared directory, integrates with runtime update
hooks, and will fill-in the `${python:Depends}', `${python:Versions}',
and `${python:Provides}' substvars. See the python-support
documentation in /usr/share/doc/python-support for details.
B.3. python-central
-------------------
python-central provides another way to manage Python modules. It
integrates with `debhelper', manages byte-compilation, private
modules, will properly use the /usr/share/pyshared directory,
integrates with runtime update hooks, and will fill-in the
`${python:Depends}', `${python:Versions}', and `${python:Provides}'
substvars. See the python-central documentation in the pycentral(1)
and dh_pycentral(1) man pages.
B.4. CDBS
---------
The CDBS python-distutils.mk class helps packaging of setup.py based
Python packages.
-------------------------------------------------------------------------------
C. Upgrade Procedure
--------------------
This section describes the procedure for the upgrade when the default
Python version is changed in the `unstable' distribution, requiring
recompilation of many Python-related packages.
1. Selected pre-releases and release candidates of new Python
versions are uploaded to Experimental to support pre-transition
work and testing.
2. Application and module maintainers make sourceful changes where
needed to prepare for the new Python version when needed.
3. Have a long and heated discussion.
4. The Debian Python maintainer and module/application maintainers
discuss the readiness for a new default Debian Python version and
associated packaging/policy changes. Once there is some
consensus, the Python maintainer announces the upgrade and
uploads to Unstable.
5. Upload of the Python core meta-packages `python', `python-dev',
`python-doc' and several `python-<module>', depending on the new
`python<X>.<Y>', `python<X>.<Y>-dev' and so on.
6. The release team schedules rebuilds for packages that may need
it. Packages that require additional manual work get updated and
uploaded.
-------------------------------------------------------------------------------
Debian Python Policy
Neil Schemenauer <nas@debian.org>
Matthias Klose <doko@debian.org>
Gregor Hoffleit <flight@debian.org>
Josselin Mouette <joss@debian.org>
Joe Wreschnig <piman@debian.org>
Loi"c Minier <lool@debian.org>
Scott Kitterman <scott@kitterman.com>
version 0.9.1.0
--- python-policy2.5.4-5.txt 2010-01-15 11:19:05.000000000 -0500
+++ python-policydraft.txt 2010-01-15 11:22:30.000000000 -0500
@@ -16,7 +16,7 @@
Scott Kitterman <scott@kitterman.com>
- version 0.9.0.0
+ version 0.9.1.0
-------------------------------------------------------------------------------
@@ -76,12 +76,15 @@
2.1. Types of Python Modules
2.2. Module Package Names
2.3. Specifying Supported Versions
+ 2.3.1. Build time determination for python2.<X> versions
+ 2.3.2. Build time determinations for python3.<X> versions
+ 2.3.3. Run Time/Binary Package Versions
2.4. Dependencies
2.5. Provides
2.6. Modules Byte-Compilation
3. Python Programs
- 3.1. Programs using the default python
+ 3.1. Programs using the default Python
3.1.1. Programs Shipping Private Modules
3.2. Programs Using a Particular Python Version
@@ -124,24 +127,29 @@
Apart from the default version, legacy versions of Python or beta
versions of future releases may be included as well in the
distribution, as long as they are needed by other packages, or as long
- as it seems reasonable to provide them. (Note: For the scope of this
- document, Python versions are synonymous to feature releases, i.e.
- Python 2.5 and 2.5.1 are sub-minor versions of the same Python version
- 2.5, but Python 2.4 and 2.5 are indeed different versions.)
+ as it seems reasonable to provide them. Python3 versions are a
+ seperate series from Python(2) versions. Unless specifically
+ mentioned, the policy for Python versions also applies to Python3
+ versions. (Note: For the scope of this document, Python versions are
+ synonymous to feature releases, i.e. Python 2.5 and 2.5.1 are
+ sub-minor versions of the same Python version 2.5, but Python 2.4 and
+ 2.5 are indeed different versions.)
For any version, the main binary package must be called
`python<X>.<Y>'.
- The set of currently supported python versions can be found in
- `/usr/share/python/debian_defaults'. This file is in Python
- ConfigParser format and defines four variables in its DEFAULT section:
- default-version which is the current default Python runtime,
- supported-versions which is the set of runtimes currently supported
- and for which modules should be built and byte-compiled, old-versions
- which is the list of runtimes which might still be on the system but
- for which should not be built anymore, and unsupported-versions which
- is the list of runtimes which should not be supported at all, that is
- modules should not be built or byte-compiled for these.
+ The set of currently supported Python versions can be found in
+ `/usr/share/python/debian_defaults'. The public interface to this
+ file is `/usr/bin/pyversions' for Python2 versions and
+ `/usr/bin/py3versions' for Python3. It includes default-version,
+ which is the current default Python runtime, supported-versions which
+ is the set of runtimes currently supported and for which modules
+ should be built and byte-compiled, old-versions which is the list of
+ runtimes which might still be on the system but for which should not
+ be built anymore, and unsupported-versions which is the list of
+ runtimes which should not be supported at all, that is modules should
+ not be built or byte-compiled for these. This file may list both a
+ default Python version and a default Python3 version.
unsupported-versions is a superset of (includes) old-versions and the
default-version is always in supported-versions.
@@ -165,16 +173,17 @@
(for example the profile module), for dependency tracking purposes
(for example the GPL-licensed gdbm module) or that should not be
included for packaging reasons (for example the tk module which
- depends on Xorg). Some tools and files for the _development_ of
+ depends on X.Org). Some tools and files for the _development_ of
Python modules are split off in a separate binary package
`python<X>.<Y>-dev'. Documentation will be provided separately as
well.
At any time, the `python' binary package must ensure that
`/usr/bin/python' is provided as a symlink to the current
- `python<X>.<Y>' executable. The `python' binary package must also
- depend on the appropriate `python<X>.<Y>' to ensure this runtime is
- installed.
+ `python<X>.<Y>' executable. `/usr/bin/python3' is to be provided
+ similarly by the `python3' binary package. The `python' binary
+ package must also depend on the appropriate `python<X>.<Y>' to ensure
+ this runtime is installed.
The version of the `python' binary package must be greater than or
equal to <X>.<Y> and smaller than <X>.<Y+1>.
@@ -206,14 +215,14 @@
The preferred specification for the Python interpreter is
`/usr/bin/python' or `/usr/bin/python<X>.<Y>'. This ensures that a
- Debian installation of python is used and all dependencies on
- additional python modules are met.
+ Debian installation of Python is used and all dependencies on
+ additional Python modules are met.
If a maintainer would like to provide the user with the possibility to
override the Debian Python interpreter, he may want to use
`/usr/bin/env python' or `/usr/bin/env python<X>.<Y>'. However this
is not advisable as it bypasses Debian's dependency checking and makes
- the package vulnerable to incomplete local installations of python.
+ the package vulnerable to incomplete local installations of Python.
1.5. Module Path
@@ -232,25 +241,27 @@
exception to the above, modules managed by python-support are
installed in another directory which is added to the sys.path using
the .pth mechanism. The .pth mechanism is documented in the Python
- documentation of the `site' module. A special directory is dedicated
- to public Python modules installed by the local administrator,
- /usr/local/lib/python<X>.<Y>/dist-packages for python2.6 and later,
- and /usr/local/lib/python<X>.<Y>/site-packages for python2.5 and
- earlier. For a local installation by the administrator of python2.6
- and later, a special directory is reserved to Python modules which
- should only be available to this Python,
- /usr/local/lib/python<X>.<Y>/site-packages. Unfortunately, for
- python2.5 and earlier this directory is also visible to the system
- Python. Additional information on appending site-specific paths to
- the module search path is available in the official documentation of
- the site module.
+ documentation of the `site' module. For Python3, the only public
+ module installation location is /usr/lib/python<X>.<Y>/dist-packages.
+ A special directory is dedicated to public Python modules installed by
+ the local administrator, /usr/local/lib/python<X>.<Y>/dist-packages
+ for python2.6 and later, and
+ /usr/local/lib/python<X>.<Y>/site-packages for python2.5 and earlier.
+ For a local installation by the administrator of python2.6 and later,
+ a special directory is reserved to Python modules which should only be
+ available to this Python, /usr/local/lib/python<X>.<Y>/site-packages.
+ Unfortunately, for python2.5 and earlier this directory is also
+ visible to the system Python. Additional information on appending
+ site-specific paths to the module search path is available in the
+ official documentation of the site module.
When binary packages ship identical source code for multiple Python
versions, for instance /usr/lib/python2.6/dist-packages/foo.py and
/usr/lib/python2.5/site-packages/foo.py, these should point to a
common file. A common location to share, across Python versions,
arch-independent files which would otherwise go to the directory of
- system public modules is /usr/share/pyshared.
+ system public modules is /usr/share/pyshared. For Python3, this
+ location is /usr/share/py3shared.
1.6. Hooks for updates to installed runtimes
@@ -325,7 +336,7 @@
linked to libpython.
Python packages are directories containing at least a `__init__.py',
- other modules, extensions and packages (A package in the Python sense
+ other modules, extensions, or packages (A package in the Python sense
is unrelated to a Debian package). Python packages must be packaged
into the same directory (as done by upstream). Splitting components
of a package across directories changes the import order and may
@@ -349,10 +360,12 @@
preferably be named `python-<foo>', if the module name allows, but
this is not required if the binary package ships multiple modules. In
the latter case the maintainer chooses the name of the module which
- represents the package the most. Such a package should support the
- current Debian Python version, and more if possible (there are several
- tools to help implement this, see Appendix B, `Packaging Tools'). For
- example, if Python 2.3, 2.4, and 2.5 are supported, the Python command
+ represents the package the most. For modules that support Python3,
+ they should be named `python3-<foo>'. Such a package should support
+ the current Debian Python version, and more if possible (there are
+ several tools to help implement this, see Appendix B, `Packaging
+ Tools'). For example, if Python 2.3, 2.4, and 2.5 are supported, the
+ Python command
import foo
@@ -360,11 +373,22 @@
`/usr/bin/python2.3', `/usr/bin/python2.4', and `/usr/bin/python2.5'.
This requirement also applies to extension modules; binaries for all
the supported Python versions should be included in a single package.
+ Namespace packages require some additional considerations. Only one
+ one package can provide a namespace (common __init__.py file) and
+ other packages that use the same namespace must depend on this
+ package. Package providing the top level namespace should be named
+ using the same rules as other Python binary module packages. If this
+ is not feasible, python-namespace-common should be used. Sub-packages
+ built from the same source should be named using namespace + package
+ separated by a period, i.e. `python-<foo>.<bar>'.
2.3. Specifying Supported Versions
----------------------------------
+2.3.1. Build time determination for python2.<X> versions
+--------------------------------------------------------
+
The optional `XS-Python-Version' field in `debian/control' specifies
the versions of Python supported by the source package. When not
specified, it defaults to all currently supported Python versions. It
@@ -385,17 +409,53 @@
package would only have to support a single version (even across
default version changes).
- Your control file should also have a line:
+ Optionally, the python3.<X> approach described below may also be used
+ via `/usr/bin/pyversions', but only in cases where generating
+ appropriate Depends and Provides lines with this approach is
+ supported. For pyversions, it will always return values less than
+ 3.0.
+
+2.3.2. Build time determinations for python3.<X> versions
+---------------------------------------------------------
+
+ Packages should be built for all currently supported Python versions
+ for which they are compatible. The defined interface for determining
+ which Python versions are supported is `/usr/bin/py3versions'. The
+ output should be used to generate appropriate Depends and Provides
+ information at package build time. The format to call py3versions is
+ the same format used specify package version requirements in
+ `debian/control' (for py3versions the lowest version it will return is
+ 3.0):
+
+ /usr/bin/py3versions ">= X.Y"
+ /usr/bin/py3versions ">= A.B, << X.Y"
+ /usr/bin/py3versions "A.B, X.Y"
+
+ The keywords "all" and "current" that were used with Python2.<X> are
+ not applicable to Python3.<X>.
+
+2.3.3. Run Time/Binary Package Versions
+---------------------------------------
+
+ Your control file must also, for each Python related binary, have a
+ line:
XB-Python-Version: ${python:Versions}
The python:Versions is substituted by the supported Python versions of
- the binary package, based on `XS-Python-Version'. (If you are not
- using python-central or python-support, you will need to handle this
- substitution yourself.) The format of the field `XB-Python-Version' is
- the same as the `XS-Python-Version' field for packages not containing
- extensions. Packages with extensions must list the versions
- explicitly.
+ the binary package, based on the build time determination described
+ above. For python2.<X> packages not containing extensions the
+ substitution may produces any output format permitted for the
+ `XS-Python-Version'. For all packages with extensions and all
+ python3.<X> packages, the expansion must list the versions explicitly.
+
+ XB-Python-Version: 3.0, 3.1
+
+ Directly hard coding XB-Python-Version values in debian/control is
+ deprecated. The primary purpose of XB-Python-Version is to support
+ automating the detection of packages needing update when the supported
+ Python versions change. Hard coded values or non-specific values like
+ all or current do not support this design goal.
If your package is used by another module or application that requires
a specific Python version, it should also `Provide: python<X>.<Y>-foo'
@@ -432,12 +492,15 @@
If a binary package provides any binary-independent modules (`foo.py'
files), the corresponding byte-compiled modules (`foo.pyc' files) and
optimized modules (`foo.pyo' files) must not ship in the package.
- Instead, they should be generated in the package's postinst, and
- removed in the package's prerm. The package's prerm has to make sure
- that both `foo.pyc' and `foo.pyo' are removed.
+ Instead, they should be generated for all Python versions at build
+ time (to ensure the byte compilation will succeed on user's systems),
+ discarded, and then generated on the installed system in the package's
+ postinst. They must be removed in the package's prerm. The package's
+ prerm has to make sure that both `foo.pyc' and `foo.pyo' for all
+ Python versions are removed.
- A binary package should only byte-compile the files which belong to
- the package.
+ A binary package must only byte-compile the files which belong to the
+ package.
The file `/etc/python/debian_config' allows configuration how modules
should be byte-compiled. The postinst scripts should respect these
@@ -458,7 +521,7 @@
------------------
-3.1. Programs using the default python
+3.1. Programs using the default Python
--------------------------------------
Programs that can run with any version of Python must begin with
@@ -466,7 +529,7 @@
preferred). They must also specify a dependency on `python', with a
versioned dependency if necessary.
- If the program needs the python module `foo', it must depend on the
+ If the program needs the Python module `foo', it must depend on the
real package providing this module, usually `python-foo' but this name
might vary when the package ships multiple modules.
@@ -522,8 +585,8 @@
Programs which embed a Python interpreter must declare a
`Build-Depends' on `python<X>.<Y>-dev', where python<X>.<Y> is the
- python version the program builds against. It should be the current
- default python version unless the program does not work correctly with
+ Python version the program builds against. It should be the current
+ default Python version unless the program does not work correctly with
this version.
@@ -545,9 +608,9 @@
As long as you don't install other versions of Python in your path,
Debian's Python versions won't be affected by a new version.
- If you install a different sub-release of the version of python you
+ If you install a different sub-release of the version of Python you
have got installed, you will need to be careful to install all the
- modules you use for that version of python too.
+ modules you use for that version of Python too.
-------------------------------------------------------------------------------
@@ -565,6 +628,8 @@
Some applications and pure Python modules may be able to build-depend
only on `python' or `python-all' and not require the -dev packages.
+ For Python3, use `python3-all-dev' or `python3-all'
+
Build-Depend on at least:
Build-Depends: python2.3 (>= 2.3-1)
@@ -588,11 +653,9 @@
------------------
This section describes the various tools to help package Python
- programs and modules for Debian. Although none of these tools are
- mandatory, their use is strongly encouraged, as the above policy has
- been designed with them in mind (and vice versa). This appendix is
- just an overview. If you use these tools, you should read their full
- documentation.
+ programs and modules for Debian. None of these tools are mandatory.
+ This appendix is just an overview. If you use these tools, you should
+ read their full documentation.
B.1. distutils
@@ -618,7 +681,7 @@
the "install" command of `setup.py' with a system-provided python2.6
or later versions, Python modules will be installed to
/usr/lib/python<X>.<Y>/dist-packages which is only seen by the
- system-provided python, not by a local installation. Using the
+ system-provided Python, not by a local installation. Using the
`--install-layout=deb' flag to `setup.py' with a system-provided
python2.4 or python2.5 does not affect the default installation
directory.
@@ -662,8 +725,8 @@
--------------------
This section describes the procedure for the upgrade when the default
- python version is changed in the `unstable' distribution, requiring
- recompilation of many python-related packages.
+ Python version is changed in the `unstable' distribution, requiring
+ recompilation of many Python-related packages.
1. Selected pre-releases and release candidates of new Python
versions are uploaded to Experimental to support pre-transition
@@ -680,7 +743,7 @@
consensus, the Python maintainer announces the upgrade and
uploads to Unstable.
- 5. Upload of the python core meta-packages `python', `python-dev',
+ 5. Upload of the Python core meta-packages `python', `python-dev',
`python-doc' and several `python-<module>', depending on the new
`python<X>.<Y>', `python<X>.<Y>-dev' and so on.
@@ -703,5 +766,5 @@
Scott Kitterman <scott@kitterman.com>
- version 0.9.0.0
+ version 0.9.1.0
Reply to: