Re: Python3 modules not built for all supported Python versions
On 30/03/2020 16:08, Simon McVittie wrote:
> On Mon, 30 Mar 2020 at 15:30:01 +0200, Johannes Schauer wrote:
>> does this mean that build-depending on python3-dev is wrong in general and
>> should instead be replaced by build-depending on python3-all-dev?
>
> It is only wrong for packages that build Python 3 extensions (binary
> modules) that are intended to be loadable by all supported Python
> 3 versions (roughly: `find /usr/lib/python3/dist-packages -name '*.so'`).
>
> For packages that embed Python 3, like the versions of vim that
> have Python scripting support, or packages that use a Python 3
> extension as an internal implementation detail of some tool, like
> gobject-introspection, my understanding is that build-depending
> on python3-dev continues to be appropriate. These extensions would
> ideally be installed in a private directory, like gobject-introspection's
> /usr/lib/x86_64-linux-gnu/gobject-introspection/giscanner/_giscanner.cpython-38-x86_64-linux-gnu.so
> - but I know some upstreams and some downstream maintainers (arguably
> incorrectly) package private extensions as though they were public
> extensions, because the mechanics of doing so are much simpler.
>
>> For example the package src:ros-geometry2 has a super simple
>> dh-style rules file, basically just doing:
>>
>> %:
>> dh $@ --buildsystem=cmake --with python3
>>
>> What would I have to change to successfully fix this problem?
>
> The general answer is that you would have to build it repeatedly in a
> loop, with each supported version of Python 3 in turn. I am not aware
> of a way to do this in a similarly simple rules file.
I've heard pybuild now has a cmake backend, so theoretically you could do
something like
%:
dh $@ --buildsystem=pybuild --system=cmake
Since I couldn't find any package in the archive that used pybuild with the
cmake backend, I have looked at this specific package and came up with the
attached debdiff. I added the dh_auto_install hack because the cmake build
system creates the python extension as _tf2.so, and is only renamed by
dh_python3 (as can be seen in current build logs). However when building for
both python3.7 and python3.8, the last installation will override the former,
and only one _tf2.so will be available.
However that workaround proved to not be enough, because dh_install3 renames
both the 3.7 and 3.8 versions as 3.8:
make[2]: Leaving directory '/build/ros-geometry2-0.6.6/.pybuild/cpython3_3.7/build'
I: pybuild pybuild:310: dh_python3
I: dh_python3 fs:343: renaming _tf2.so to _tf2.cpython-38-x86_64-linux-gnu.so
[...]
make[2]: Leaving directory '/build/ros-geometry2-0.6.6/.pybuild/cpython3_3.8/build'
I: pybuild pybuild:310: dh_python3
W: dh_python3 fs:340: destination file exist, cannot rename _tf2.so to
_tf2.cpython-38-x86_64-linux-gnu.so
Resulting in these installed files:
-rw-r--r-- root/root 69112 2020-03-30 14:56
./usr/lib/python3/dist-packages/tf2_py/_tf2.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- root/root 69128 2020-03-30 14:56
./usr/lib/python3/dist-packages/tf2_py/_tf2.so
I don't know if I'm missing an argument to dh_python3 so that it knows the
python version, or even if there's a better workaround. But perhaps pybuild
should be doing this automatically between the dh_auto_install calls so that
this kind of workarounds aren't necessary.
Piotr, is this a bug in pybuild, or am I doing something wrong?
Cheers,
Emilio
diff -Nru ros-geometry2-0.6.6/debian/changelog ros-geometry2-0.6.6/debian/changelog
--- ros-geometry2-0.6.6/debian/changelog 2020-01-18 21:51:17.000000000 +0100
+++ ros-geometry2-0.6.6/debian/changelog 2020-03-30 16:56:18.000000000 +0200
@@ -1,3 +1,10 @@
+ros-geometry2 (0.6.6-1.1) UNRELEASED; urgency=medium
+
+ * Non-maintainer upload.
+ * Build for all supported versions of python3.
+
+ -- Emilio Pozuelo Monfort <pochu@debian.org> Mon, 30 Mar 2020 16:56:18 +0200
+
ros-geometry2 (0.6.6-1) unstable; urgency=medium
* New upstream version 0.6.6
diff -Nru ros-geometry2-0.6.6/debian/control ros-geometry2-0.6.6/debian/control
--- ros-geometry2-0.6.6/debian/control 2020-01-18 21:51:17.000000000 +0100
+++ ros-geometry2-0.6.6/debian/control 2020-03-30 16:56:18.000000000 +0200
@@ -6,7 +6,7 @@
Leopold Palomo-Avellaneda <leo@alaxarxa.net>
Build-Depends: debhelper-compat (= 12), dh-exec,
catkin (>= 0.7.14-4), libroscpp-core-dev, ros-message-generation, libstd-msgs-dev,
- python3-dev, python3-setuptools, dh-python,
+ python3-all-dev, python3-setuptools, dh-python,
libgeometry-msgs-dev, ros-actionlib-msgs,
libconsole-bridge-dev, python3-rospy (>= 1.14.3+ds1-7), python3-rosgraph (>= 1.14.3+ds1-7),
libactionlib-dev, librosconsole-dev,
diff -Nru ros-geometry2-0.6.6/debian/rules ros-geometry2-0.6.6/debian/rules
--- ros-geometry2-0.6.6/debian/rules 2019-10-25 17:04:01.000000000 +0200
+++ ros-geometry2-0.6.6/debian/rules 2020-03-30 16:56:18.000000000 +0200
@@ -4,9 +4,10 @@
export ROS_PYTHON_VERSION=3
+export PYBUILD_CONFIGURE_ARGS="-DCATKIN_ENABLE_TESTING=OFF"
+
%:
- dh $@ --buildsystem=cmake --with python3
+ dh $@ --buildsystem=pybuild --with python3 -- --system cmake
-override_dh_auto_configure:
- dh_auto_configure -- \
- -DCATKIN_ENABLE_TESTING=OFF
+override_dh_auto_install:
+ dh_auto_install -- --after-install dh_python3
Reply to: