Re: Bug#133306: apt-listchanges: Does not handle .pyc files correctly
On Tue, Feb 19, 2002 at 10:16:09AM +0100, Christian Kurz wrote:
> On 18/02/02, Bastian Kleineidam wrote:
> > On Mon, Feb 18, 2002 at 12:48:02PM -0500, Matt Zimmerman wrote:
[...]
> > Look at http://people.debian.org/~calvin/python-central/
OK, I promised I would have a look are python-central and return comments.
Here they are;
First, remember that this tool is explicity for the subset of packages
containing pure-python modules that work with multiple versions of Python.
The existing policy already provides ways of handling all cases, but this
tool allows more efficient support of multiple python versions for this
particular subset of packages.
In general, this is a neat idea. The seperation into a seperate package
makes this nice and optional from a module-packager's point of view, but
will require support in each pythonX.Y package for it to work cleanly. I
feel irrationaly uncomfortable about introducing an emacs-like
module-registry, but cannot claim to have a better way to do it. I'm slowly
becoming more concerned about blindly supporting "all" or ">=X.Y", but this
does allow packagers to support only particular versions by explicitly
specifying them.
Some things like how dependancies should work need to be resolved. The
sample "sperm" package has a dependancy on "python", but this doesn't really
express it right. This means a dependancy on any version of the "default"
python. This means "sperm" cannot be installed for python1.5 unless
"python", and hence "python2.1", is also installed. A dependancy on
"python1.5 | python1.6 | python2.0 | python2.1 | python2.2" would work
better, but then it breaks for python2.3, undermining it's claimed
python-central support for "all".
This leads to some of my concerns about claiming support for "all" or
">=X.Y". As one of the people who originaly provided nasty-hack code to
support this, I'm starting to have doubts. For starters, claming support for
as-yet-non-existant versions of Python is more risky than I thought. I would
hesitate to deny a packager the right to make such claims and deal with the
bug-reports when they come in, but I think it should be discoraged.
Looking at the code for register-python-package, support for "all" and
">=X.Y" makes it more complex for little gain. There is support for
registering and unregistering modules incrementaly for various versions of
python, but it behaves strange for incremental registration/unregistration
of "all" or ">=X.Y". I can't see how this could be useful.
The implementation of python-central is a little buggy. The explicit version
support is broken in get_versions() because it doesn't check what versions
of python are installed. This means module-configure() will attempt to
compile for non-installed versions of python. Also I'm not sure about the
safety of doing "cat $FILE | grep 'something' > $FILE", but it seems to
work.
There is redundancy in the code with duplication between the python and
module configure/remove routines. Registered packages will be compiled twice
when a version of python is re-configured because compileall.py is used
before registered packages are compiled.
In conclusion, I think that python-central support for "all" and ">=X.Y" is
problematic, and should be removed. Even without this support, packagers can
dance with disaster by explicitly specifying all versions up to 5.0 if they
want.
In order to put my code where my mouth is, I've attached a modified version
of register-python-package. I have factored common code into subroutines and
removed support for "all" and ">=X.Y". I have fixed get_versions() to detect
installed python versions. I have left in incremental register/unregister
support, and not fixed module compilation twice on python re-configure. I
have also made the checking of parameters a bit more robust, and tidied the
code a little.
The correct way to use this modified version of register-python-package is
as follows;
Register for specific versions 2.1 and 2.2:
1) Build-Depends: python2.1-dev, python2.2-dev
(Note: is it strictly necisary to list all -dev versions?)
2) Depends: (python2.1 | python2.2), python-central
3) In the postinst script call
register-python-package module configure <package name> 2.1 2.2
4) In the prerm script call
register-python-package module remove <package name> 2.1 2.2
Those who want "all" behaviour can specify "1.5 1.6 2.0 2.1 2.2 2.3". Those
who want ">=X.Y" behaviour can just specify "X.Y X.Y+1 ..." upto wherever
they feel comfortable with.
--
----------------------------------------------------------------------
ABO: finger abo@minkirri.apana.org.au for more info, including pgp key
----------------------------------------------------------------------
#!/bin/sh
# the python registrar
# directory with registered packages
CENTRAL=/var/lib/python-central
# python command to byte-compile a file
BYTECOMP="import sys,py_compile;py_compile.compile(sys.argv[1],sys.argv[2])"
# directory for installed .py files
IN=/usr/lib/python/site-packages
# get_versions {<python version>...}
# return installed versions of python from the list provided
get_versions () {
RET=""
for V1 in $*; do
if [ -d /usr/lib/python$V1 ]; then
RET="$RET $V1"
fi
done
}
# module_compile <package name > <python version>
# symlinks and compiles a package for the specified python version
module_compile () {
PACKAGE=$1
VERSION=$2
PYTHON=/usr/bin/python$VERSION
OUT=/usr/lib/python$VERSION/site-packages
# look for .py files in the package
for i in `dpkg --listfiles $PACKAGE | grep "^$IN/.*\.py$" | sed "s#^$IN/##"`; do
# install potentially missing (sub)directories
DIRNAME=`dirname $OUT/$i`
install -d -o root -g root -m 0755 $DIRNAME
# make relative link
REL="../.."
while [ $DIRNAME != $OUT ]; do
REL=../$REL
DIRNAME=`dirname $DIRNAME`
# prevent infinite loops
if [ "$DIRNAME" = "/" -o "$DIRNAME" = "." ]; then
exit 1
fi
done
# REL points to /usr/lib now
ln -sf $REL/python/site-packages/$i $OUT/$i
# byte-compile package .py files
$PYTHON -O -c "$BYTECOMP" $OUT/$i $OUT/${i}o
$PYTHON -c "$BYTECOMP" $OUT/$i $OUT/${i}c
# fix output file mode
chmod 0644 $OUT/${i}[co] || true
done
}
# module_clean <package name> <python version>
# removes symlinks and bytecode of a package for the specified python version
module_clean() {
# remove symlink and byte-compiled files
PACKAGE=$1
VERSION=$2
OUT=/usr/lib/python$VERSION/site-packages
for i in `dpkg --listfiles $p | grep "^$IN/.*\.py$" | sed "s#^$IN/##"`; do
rm -f $OUT/$i $OUT/${i}c $OUT/${i}o
done
# remove directories
for i in `dpkg --listfiles $p | grep "^$IN/" | sort -r | sed "s#^$IN/##"`; do
if [ -d $OUT/$i ]; then
rmdir $OUT/$i || true
fi
done
}
# module_configure <package name> { <python version>...}
# configure a module for all or specific python versions
module_configure () {
PACKAGE=$1
shift
get_versions $*
versions=$RET
# byte-compile versions
for pyver in $versions; do
module_compile $PACKAGE $pyver
done
# register this package (only the given versions)
register_module $PACKAGE $*
}
# module_remove <package name> { <python version>...}
# remove a previously configured module for all or specific python versions
module_remove () {
PACKAGE=$1
shift
get_versions $*
versions=$RET
for pyver in $versions; do
module_clean $PACKAGE $pyver
done
# unregister this package (only the given versions)
unregister_module $PACKAGE $*
}
# python_configure <python version>
# byte-compile this python version plus all registered packages
python_configure () {
pyver=$1
PYTHON=/usr/bin/python$pyver
PYLIBS=/usr/lib/python$pyver
# byte-compile all python files
$PYTHON -O $PYLIBS/compileall.py -q $PYLIBS
$PYTHON $PYLIBS/compileall.py -q $PYLIBS
# byte-compile registered packages
OUT=/usr/lib/python$pyver/site-packages
for p in `ls $CENTRAL/*.package 2>/dev/null`; do
# check if package p is registered for our python version
if `cat $p | grep "^version:$pyver\$" > /dev/null 2>&1`; then
# byte-compile the module for our python version
p=`basename $p | sed 's#.package$##'`
module_compile $p $pyver
fi
done
}
# python_remove <python version>
# remove byte-compiled files for this python version and for all
# registered python packages
python_remove () {
pyver=$1
PYTHON=/usr/bin/python$pyver
PACKAGE=python$pyver
# remove byte-compiled files for this python version
dpkg --listfiles $PACKAGE |
awk '$0~/\.py$/ {print $0"c\n" $0"o"}' |
xargs rm -f >&2
# remove registered packages
OUT=/usr/lib/python$pyver/site-packages
for p in `ls $CENTRAL/*.package 2>/dev/null`; do
# check if package p is registered for our python version
if `cat $p | grep "^version:$pyver\$" > /dev/null 2>&1`; then
# byte-compile the module for our python version
p=`basename $p | sed 's#.package$##'`
module_clean $p $pyver
fi
done
}
# register_module <package name> { <python version>...}
# register all given python versions of the package
register_module () {
PACKAGE=$1
shift
# make .package suffix to enable the placement of other files in
# this directory
FILE=$CENTRAL/$PACKAGE.package
touch $FILE
for v in $*; do
# check if version is already there
if ! `cat $FILE | grep "^version:$v\$" > /dev/null 2>&1`; then
# register new version
echo "version:$v" >> $FILE
fi
done
}
# unregister_module <package name> { <python version>...}
# unregister all given python versions of the package
unregister_module () {
PACKAGE=$1
shift
FILE=$CENTRAL/$PACKAGE.package
if [ -f $FILE ]; then
# remove versions
for v in $*; do
cat $FILE | grep -v "^version:$v$" > $FILE
done
# if empty, remove file
if ! `cat $FILE | grep '[[:alnum:]]' > /dev/null 2>&1`; then
rm -f $FILE
fi
fi
}
usage () {
echo "usage: $0 python {configure|remove} <python version>"
echo "usage: $0 module {configure|remove} <package name> \\"
echo " { <python version>...}"
exit 1
}
###############################################################
################ main ##########################
###############################################################
type=$1
action=$2
if [ "$type" = "module" ]; then
if [ "$action" = "configure" ]; then
shift; shift
module_configure $*
elif [ "$action" = "remove" ]; then
shift; shift
module_remove $*
else
usage
fi
elif [ "$type" = "python" ]; then
if [ "$action" = "configure" ]; then
python_configure $3
elif [ "$action" = "remove" ]; then
python_remove $3
else
usage
fi
else
usage
fi
Reply to: