On Thu, Sep 28, 2017 at 08:27:20AM -0400, Antoine Beaupré wrote: > > And moving Python 3 packages to /usr/bin/sphinx3-build or something like > > that will mean diverging from upstream (see below). > > Note that we already diverge from upstream in numerous places in Python, > first and foremost in the naming of the Python binary itself. How do we diverge there? In my opinion we follow PEP 394 quite closely: https://www.python.org/dev/peps/pep-0394/ > > Good suggestion, I have submitted a pull request upstream: > > https://github.com/sphinx-doc/sphinx/pull/4092 > > Excellent - I meant to do that but ran out of time writing the email in > the first place. :) > > I see, however, it's not getting too much traction.. But they have a > fair point - I didn't realize there was a difference between variables > from the environment and from the commandline in Make. Maybe it's good > enough as it is then. It is not ‘they’, it is our Simon :) He also replied on this mailing list, just did not CC you. > > And there is no such thing as --with=sphinx. There is --with=sphinxdoc, > > but it operates on already built and installed documentation, so it cannot > > help with building. > > I guess this is what I mean should be automated better. Every time I > used --with=sphinxdoc, I expected it to build the docs and then > remembered, when the produced package lacked any docs whatsoever, that I > needed to build things myself. > > A simple --with=sphinxdoc should be sufficient to build sphinx docs. We > shouldn't expect upstream to do that step in setup.py, as it is rarely > hooked into the main build. Yes, there's the `build_sphinx` command, but > it's not hooked into `setup.py build` except in some rare project. Sphinxdoc is a helper to run after install, not a build system. So it is not a place for such code. > This is why I mention pybuild: maybe *that* is where docs should be > built? I am not sure. In any case, I feel there's a shim missing here, > and there's a good occasion to leverage the automation we have to > generate proper build deps and build-time commands. Maybe pybuild can do this, but it then needs to make some assumptions about the doc source path, build path, wished formats, etc. For finding source path it can probably use something like this: https://github.com/sphinx-doc/sphinx/blob/stable/sphinx/setup_command.py#L111 But it is up to pybuild maintainer (Piotr) to decide whether he wants this feature or not. > > What is the problem with using /usr/share/sphinx/scripts/python3/sphinx-build > > explicitly if you want to force Python 3? > > It's hard to find. It's unusal for a user-facing command to be absent > from the path yet to expect users to find it, especially when one > version *is* available in the path. As I said ealier, for most cases just using sphinx-build which is on PATH should be fine. In advanced cases, it should not be hard to run ‘dpkg -L python3-sphinx’ to figure out where its scripts are, or use the -m syntax. > Sure. I just found out about this while writing the email, after almost > a decade of maintaining python packages. That's a clear symptom of bad > discovery, although the upstream Makefile changes may help with that > eventually. Actually quite many packages tend to provide the -m syntax and even prefer it to scripts in /usr/bin/. Examples from packages I maintain: -m markdown, -m keyring, -m nose. > > Also there is no such thing as sphinx3-build upstream (unlike other packages > > that follow this convention). So all upstream projects will try to use > > sphinx-build, not sphinx3-build, even if they are Python 3 only. And if you > > override this command anyway, you can use two existing ways, I don’t see a > > need for the third one. > > The reasoning here is that is is more discoverable, namely through > commandline completion, and is consistent with the /usr/bin/python* > binary naming conventions. I have just figured out that there *is* bash completion for python3 -m syntax. It is not difficult for me to provide the sphinx3-* binaries, but that might create false assumptions for developers that it is available on other systems too. It is better to use the -m syntax because it is more universal and works on any OS. > > I won’t say that this convention is usual. There are many packages named > > python-* or python3-* which provide executable scripts. > > I would argue that is a strange practice: I tend to ignore packages that > start with python* unless I am programming something. As a user, they > are basically useless because I am supposed to code something to use > those libraries. Now, granted, sphinx is a little special because it > *is* a programmer-oriented library, but keep in mind it can be used for > non-programs as well (see the Debian policy for example). > > > Let me summarize what you, in my opinion, should do as a maintainer of > > packages using Sphinx: > > > > 1) If your project is not using autodoc: just build-depend on python3-sphinx > > and use /usr/bin/sphinx-build. There is no need to care about > > versions. > > 2) If your project uses autodoc, and is compatible with both Python 2 and 3: > > same as in 1), just use /usr/bin/sphinx-build. > > 3) If your project uses autodoc, and is only compatible with Python 3: > > use ‘python3 -m sphinx’ or ‘python3 setup.py build_sphinx’, depending on the > > upstream build system. > > 4) If your project uses autodoc, and is only compatible with Python 2: > > Port it to python 3. Or use ‘python2 -m sphinx’ or ‘python2 setup.py > > build_sphinx’. > > > > If you think this classification is incomplete, then please let me know > > where I am wrong. > > That is a fair description, of course. It would be great to have build > step automated. At the very least, the above should be clearly > documented somewhere, maybe in dh_sphinxdoc's README or > manpage. Documenting that dh_sphinxdoc doesn't build in its manpage > would also go a long way, as that rather implicit right now. (ie. it > doesn't say it builds sphinx docs, but it doesn't say it doesn't either: > we need to assume that the list of things it does is exhaustive and > complete, something I never assume about documentation :) Now it is explicit: https://anonscm.debian.org/cgit/python-modules/packages/sphinx.git/commit/?id=5b2efffcaae8c915 > Okay then the policy seems to point towards shipping sphinx3-build > binary, no? See what I said above (about false assumptions). > > This may happen when setup.py depends on Sphinx and automatically calls it > > during build. Such packages have to build-depend on both Python 2 and 3 > > versions of Sphinx, even if only one version of documentation actually gets > > shipped. > > I don't follow. Why do they need to depend on both? We don't need *nor* > want to build the documentation twice - if upstream's setup.py > automatically calls build_sphinx, then this is something we should be > working around (again, pybuild?) if only to save time on the buildds. I do not say that these packages do the right thing. I just tried to find a reason why they may be doing this. It might be better to ask maintainers of these packages for more precise reason. > So maybe there's just nothing else to do here in the short term. I'll > start using "make SPHINXBUILD='python3 -m sphinx'" even though I find > that real clunky. In fact, the more I think about it, the less I see the > point of using the Makefiles at all from debian/rules: i'll probably > just call "python3 -m sphinx" or "setup.py build_sphinx" myself from > there, as needed. Right, there is no reason for Makefiles. If the build process is not integrated in upstream’s main build system, there should be no difference for you which line to write in debian/rules. > The problem, then, of course, is that I lose that precious > automation. Most Python Debian packages really don't need that much > stuff in debian/rules - this is enough for 90% of the cases: > > %: > dh $@ --with=python2,python3 --buildsystem=pybuild > > Yet, when I want to build docs in there, i need to override the build > targets immediately: > > override_dh_auto_build: > dh_auto_build > # builder=html,man only supported in Sphinx 1.6, not yet released: > # http://www.sphinx-doc.org/en/master/setuptools.html > make -C doc html man SPHINXBUILD="python3 -m sphinx" > # ^untested > > I'd like to be able to just assume sphinxdoc will do the right thing and > build me a set of HTML docs (at least). Ideally, I'd be able to choose > the builders (html, man, epub, etc) through the environment, but then > that'd need to change the build-deps as well, something I understand is > basically impossible. ;) See what I said above about pybuild. > Anyways, I learned a lot through this. Thanks for all the information, > and I hope we can update docs somewhere to reflect the current situation > and move ahead with some better build process for this as well. -- Dmitry Shachnev
Attachment:
signature.asc
Description: PGP signature