providing sphinx3-* binaries
Hi!
[please keep me in CC, I am not on the list]
We just had a short conversation on the #debian-devel IRC channel
regarding the upcoming Python 2 EOL, in the context of the Sphinx
packages.
As a packager of Python tools (and an upstream), I find the current
situation a little confusing. My Python 2 workflow is okay, but I am
trying, for new projects, to use only Python 3, which gives me a rather
broken workflow. It looks something like this:
1. compile documentation using `make html`, fail with mysterious
errors.
2. realize those are because Sphinx uses Python 2 by default
3. install python3-sphinx
4. try to compile again, same errors
5. try to find the sphinx binary (generally, `dpkg -L python3-sphinx |
grep bin`, which fails, then `dpkg -L python3-sphinx | grep build`)
6. hack the Makefile to make the `SPHINXBUILD` variable overridable
(with `?=`)
7. fix my docs and debian/rules to call:
`SPHINXBUILD=/usr/share/sphinx/scripts/python3/sphinx-build`
So that's the first problem I have. The other problem we identified is
that the /usr/bin/sphinx-build symlink is not owned by any package.
$ LANG=C dpkg -S /usr/bin/sphinx-build
dpkg-query: no path found matching pattern /usr/bin/sphinx-build
That's a rather bad practice... We shouldn't install those symlinks in
postinst. They should be alternatives or conflicts or *something*. In
2013, the sphinx maintainer [explained][1] the following:
> True for docutils; however, sphinx doesn't use alternatives, and it
> doesn't do so for good reasons.
>
> The alternatives mechanisms is only suitable if both commands are
> compatible, i.e. their behavior doesn't vary with Python version. This
> is the case for rst2html and friends, but no so much for
> sphinx-build. The problem is that sphinx-build can import 3rd-party
> Python code, which is not necessarily compatible with both Python 2
> and 3.
>
> As I understand it, python{2,3}-coverage are NOT compatible, and
> therefore they should NOT use alternatives.
I understand that reasoning, but I don't think I agree with it. Or at
least, I don't think that, in the long term, it should be something that
blocks us.
I am not sure what the solution is, but I would offer the following:
1. there should be a more easy way to override SPHINXBUILD in
quickstart-generated Makefiles. IMHO, that means SPHINXBUILD?=
instead of SPHINXBUILD=
2. similarly, --with=sphinx should do the right thing depending on the
detected pybuild Python version, that is, call
/usr/share/sphinx/scripts/python{,3}/sphinx-build directly depending
on which version we are building for.
3. sphinx packages should follow the "python vs python3"
convention. right now, when you run the "python" binary, you get
Python 2. if you want Python 3, you run "python3". I don't know if
there are plans to change this, but that's irrelevant at this stage:
sphinx should follow convention and call python2 sphinx when you run
"sphinx-build" and python3 sphinx when you run "sphinx3-build", and
both should be available in $PATH.
4. the sphinx package is not following the usual convention of
providing only libraries in python*. or to be more precise, it's
rather odd that "sphinx-common" is what installs the python2 or
python3 binary in $PATH. i would suggest providing new binary
packages called "sphinx" and "sphinx3" that would provide those
binaries.
5. optionally, the sphinx and sphinx3 packages *may* conflict with each
other to fight over the sphinx-build symlink. then people needing to
build against *both* packages can use the "python -m sphinx"
mechanism in their makefiles, since it's what upstream seems to be
[doing][2]
6. there are currently packages depending on *both* python-sphinx and
python3-sphinx. for me, that makes no sense at all: there is a
single set of documentation files built in a package, and it must be
on either one of those packages, but not both. if the software
supports building with python3, then it should depend on
python3-sphinx, if not, it should depend on python-sphinx. by
switching to sphinx{,3} binary packages, this would make that
distinction clearer as well.
My pain points would mostly be fixed with th e simple #1 and #3
fixes. Without breaking anything, we could already provide a
sphinx3-build binary, and we could file a PR upstream to make
SPHINXBUILD more easily overridable. The rest is just a brainstorm of
possible medium-term fixes to try and fix that dangling symlink issue.
I understand this is a complex situation and I would love to have better
answers, but I'm not sure there are any simple answers here. We
shouldn't hesitate to make radical changes, however: we have transition
tools to help us with that stuff, and if we can upgrade libc and gcc
across all of Debian fairly painlessly these days, i don't see why we
couldn't articulate such a transition for ~1000 python packages. :)
Thank you for your time.
A.
[1]: https://lists.debian.org/debian-python/2013/10/msg00048.html
[2]: https://github.com/sphinx-doc/sphinx/pull/3523
Reply to: