[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

pylibs and pydeps: shlibs-like support for Python extensions



The immediate problem: PyGTK encourages other Python modules to use its
code generator and GObject wrappers. Between 2.6 and 2.8 the ABI changed
so that extensions build against 2.8 do not work against 2.6. This is
breaking gst-python and its reverse dependencies, soundconverter,
quodlibet, and istanbul. It may break other packages as well.

The general problem: Python modules that provide build-time shared
objects have no way of expressing dependencies like this. PyGTK is the
first example to break, probably because it's very large and widely
used. However, it's not unique in this respect: Pygame uses shared
objects from Numeric, for example.

The solution: Debian solved this problem for regular shared objects with
shlibs files. So the attached changes to dh_python and Python policy are
an implementation of something like shlibs files, for Python.

Open questions:
 * Right now, the pylibs files are stored in /var/lib/dpkg/info. If
   the dpkg maintainers don't like this, they can easily be moved
   elsewhere, but this seemed like a good place given their analogous
   purpose to shlibs files.
 * I've not written any debhelper code before, and so my patch might
   have a dumb mistake in it. My Perl is also rusty.
 * See the note on package aliases in the policy update.
-- 
Joe Wreschnig <piman@debian.org>
--- /usr/bin/dh_python	2006-01-16 16:36:44.000000000 -0600
+++ dh_python	2006-01-31 16:03:41.000000000 -0600
@@ -18,7 +18,11 @@
 
 dh_python is a debhelper program that is responsible for generating the
 ${python:Depends} substitutions and adding them to substvars files. It
-will also add a postinst and a prerm script if required.
+will also add a postinst and a prerm script if required, and install a
+pylibs file if one is present.
+
+If you are using a pydeps module list, it will also fill ${python:Depends}
+with the appropriate versioned dependencies from their pylibs files.
 
 The program will look at python scripts and modules in your package, and
 will use this information to generate a dependency on python, with the
@@ -109,6 +113,11 @@
 	s#^/##;
 }
 
+# pylibs support
+# FIXME: This may not be the best place; it also requires approval
+# from the dpkg developers.
+use constant PYLIBS_DIR => '/var/lib/dpkg/info';
+
 # dependency types
 use constant PROGRAM   => 1;
 use constant PY_MODULE => 2;
@@ -210,17 +219,54 @@
 		}
 	}
 
+	# Install a pylibs file, if we have one.
+	if (my $pylibs_file = pkgfile($package, "pylibs")) {
+		doit("install", "-D", $pylibs_file,
+		     tmpdir($package) . "/DEBIAN/pylibs");
+	}
+
+	# Find potential pylibs dependencies
+	my %pylibs_modules = ();
+	if (my $pydeps_filename = pkgfile($package, "pydeps")) {
+		open PYDEPS_LIST, $pydeps_filename;
+		foreach my $pylibs_pkg (<PYDEPS_LIST>) {
+			chomp $pylibs_pkg;
+			my $pylibs_file = PYLIBS_DIR . "/$pylibs_pkg.pylibs";
+			if (-f $pylibs_file) {
+				open(PYLIBS_FILE, $pylibs_file);
+				my @pylibs_deps = <PYLIBS_FILE>;
+				chomp @pylibs_deps;
+				$pylibs_modules{$pylibs_pkg} = \@pylibs_deps;
+			}
+		}
+	}
+
 	# Dependencies on current python
 	$dep_on_python = 1 if $deps;
 	$strong_dep = 1 if($deps & (PY_MODULE|SO_MODULE));
 
 	if ($dep_on_python) {
 		addsubstvar($package, "python:Depends", $python, ">= $python_version");
+
 		if ($strong_dep) {
 			addsubstvar($package, "python:Depends", $python, "<< $python_nextversion");
 		} else {
 			addsubstvar($package, "python:Depends", $python, "<< $python_nextmajor");
 		}
+
+		# pylibs dependencies are meaningless without a shared
+		# object module, but if they've included a pydeps file,
+		# use them anyway.
+		for my $pylibs_pkg (keys %pylibs_modules) {
+			my $r_pylibs_versions = $pylibs_modules{$pylibs_pkg};
+			my @pylibs_versions = @$r_pylibs_versions;
+			foreach my $pylibs_ver (@pylibs_versions) {
+				if ($pylibs_ver) {
+					addsubstvar($package, "python:Depends",
+						    $pylibs_pkg, $pylibs_ver);
+				}
+			}
+		}
 	}
 
 	my $need_prerm = 0;
3.3. Extensions Used By Other Extensions
----------------------------------------
If your package contains compiled Python extensions that other
extensions use, or otherwise creates a versioned dependency at
build-time, it should provide pylibs files. A pylibs file is similar
to an shlibs file; it provides version dependency information for
other packages at build time.

The pylibs file format is one version clause per line, e.g. the
contents of /var/lib/dpkg/info/python-gtk2.pylibs might be:
	 >= 2.8
	 << 2.10

If your package builds extensions that depend on other binary
extensions, or has dependencies otherwise affected by the versions of
installed modules at build-time, you should use a pydeps file. If you
are using dh_python, you can make one named debian/pydeps or
debian/<package>.pydeps and it will be automatically used and the
dependencies substituted as part of ${python:Depends}. The format of a
pydeps file is one binary package name per line; all binary packages
with such a build-time dependency should be listed. The dependency
information will be read from their pylibs files.

Because Python module dependencies cannot be automatically detected
like shlibs files, all dependencies must be explicitly declared. So,
if your package Build-Depends on python2.3-gtk2 it must have
python2.3-gtk2 in its pydeps file, not python-gtk2. Likewise,
python2.3-gtk2 must provide its own python2.3-gtk2.pylibs file.

FIXME: Should there be a way to declare a pydeps "alias", so
python2.[234]-foo all can share the python-foo pylibs file?

pylibs and pydeps files should not be used to express API/ABI changes
in Python interfaces. Normal dependency-handling methods like virtual
packages or changing package names should be used in this
case. Additionally, they should not be used to express dependencies on
particular Python versions.

Attachment: signature.asc
Description: This is a digitally signed message part


Reply to: