Re: Questions about the Debian Python Policy
On Sat, 2005-10-22 at 22:27, James A. Treacy wrote:
> I have some questions relating to python packages and the python
> policy.
>
> I maintain a pure python program (gramps) that relies heavily on other
> python packages: python-gnome2, python-glade2, python-reportlab and
> python-gnome2-extras.
>
> Section 3.1 of the python policy states that programs which can use
> any version of python which depend on python module Foo should depend
> on python-foo, not python<X>.<Y>-foo. This can be problematic for the
> following reason.
Actually, they should have "Depends: python (>=X.Y), python (<<X.Y+1)"
where X.Y is the current default version of python.
> Let's use gramps(*) as an example and that the default python switches
> to 2.4. A user upgrades python (leaving 2.3 on the system), gramps and
> python-glade2 to python 2.4 versions but does not ugrade python-gnome2
> (this works since python 2.3 is still installed). All the dependencies
> will be met but gramps will not work as it will not find all the
> required (2.4 based) dependencies.
This should not work because the old version of python-gnome2 should
have "Depends: python (>=2.3), python (<<2.4)". This means it cannot be
installed alongside python (2.4). When you install python (2.4) you will
be forced to upgrade python-gnome2 to use the new version that has
"Depends: python (>=2.4), python (<<2.5)".
> How do you propose avoiding this situation without having programs
> depend on python<X>.<Y>-foo packages explicitly?
With the versioned dependency on python...
> Second question. Gramps installs its private modules in
> /usr/share/gramps instead of /usr/lib/site-python/gramps as specified
> in the policy. Is this ok? If not, why?
It is OK, provided you address the following issues;
1) how does gramps applications find these private modules?
2) how do you ensure that these private modules are re-compiled when
python (X.Y) upgrades to python (X.Y+1)?
> Third question. The examples for compiling python modules in
> the postinst use a specific version of python. Since gramps is
> compiled against the default verson of python, is it ok to just use
> PYTHON=python?
Yes, but you need to make sure that a re-compile is triggered when
python itself gets upgraded.
The brute-force way, as described in the current policy, is to have
"Depends: python (>=X.Y), python (<<X.Y+1)", which will force you to
release a new version of your package with new dependencies when python
upgrades... installing a new version of the package will re-compile the
py's.
Using tight version constraints to force package upgrades just to
trigger re-compiles is very ugly though. Sure, if you had binary
extensions that needed to be re-built anyway, it is no extra work, but
for a pure python package it feels wrong.
The other alternative is to have the python package trigger the compiles
for you when it is installed. Currently, the python package does no
postinst compiles at all. The pythonX.Y packages will only compile
everything in /usr/lib/python2.3. So you can't do it.
I've been advocating the brute force solution of having python's
post-inst trigger the postinst scripts of every package that depends on
python. This way packages that depend on python can re-compile their own
*.py's when python is upgraded. So far, no-one has responded...
Some packages (python-roman, pychecker, python-epydoc, apt-listchanges,
python-docutils) have taken to putting version independant pure-python
modules in /usr/lib/site-python. This is on every python versions path,
so this makes the modules available to every python version. However, it
has several problems;
1) the *.pyc's can only be compiled for one version of python. When you
use a different version of python, the pyc's will be re-compiled. If the
user has write access to /usr/lib/site-python, the old pyc's will be
overwritten.
2) which version of python do you use? You should use the version of
python that corresponds with your package's dependencies (usually
python).
3) if you compile using /usr/bin/python, how do you recompile when
python upgrades? Right now you can't.
It seems all of the packages using /usr/lib/site-python have a different
approach, and usually get it wrong in some way.
For example, pychecker has "Depends: python2.3 | python2.2 | python2.1 |
python" and it's postinst uses the first found in "python python2.3
python2.4 python2.2 python2.1". Its dependencies and what it actually
uses are different, and hence wrong. The package maintainer is at least
aware of the issues, and has a big comment in his postinst about it.
python-epydoc is similar to pychecker, but has "Depends: python2.3 |
python2.2-xmlbase | python2.1-xmlbase", so it relies on indirect
dependencies for python2.2 and python2.1. However, it too will attempt
to compile with the first found in "python python2.3 python2.4 python2.2
python2.1".
python-roman has "Depends: python (>= 2.1)", but its postinst is
hardcoded to use python2.3. This package will break if you try to
install it without python2.3 installed, but it's dependencies don't show
this. Note that it re-compiles everything in /usr/lib/site-python when
it is installed, so every other package that uses this location will
also be re-compiled using python2.3.
python-docutils is similar to python-roman, but also compiles the
private modules in /usr/share/python-docutils with python2.3, and has
"Depends: python (>=2.2), python (<<2.5)".
apt-listchanges has "Depends: python (>=2.3)" and doesn't bother
compiling anything at all, probably relying on the fact that apt is
typically run by root to ensure that the *.pycs get compiled and saved.
Its postinst however does ensure that it's one *.pyc does get removed
when the package is removed.
IMHO, python-docutils and python-roman should be compiling modules with
python, not python2.3. They will have the problem that the modules will
not be recompiled when python upgrades until the python package is
changed to trigger reconfigure's of any package that depends on python.
They will also always be vulnerable to recompiles when used by the
non-default version of python, but at least their dependencies don't
claim to support them.
pychecker and python-epydoc have the harder task of trying to support
multiple different versions of python. It will always have the problem
that pyc's can be compiled only for one version of python. The only way
to avoid this is the "documented but unsuported" method of putting
simlinks in /usr/lib/pythonX.Y for each supported pythonX.Y version.
What it currently uses is probably the best available compromise, except
they should be fixed so the compiler search list matches the
dependencies. python-epydoc should also directly depend on python if it
is to ever benefit from python triggering reconfigures of dependant
packages on install.
--
Donovan Baarda <abo@minkirri.apana.org.au>
Reply to: