Re: PEP 453 affects Debian packaging of Python packages

On 20 September 2013 01:11, Thomas Goirand <zigo@debian.org> wrote:

> From a developer point of view: this leaves you dependent on other
> people to get the latest release of your software to users, which can be
> very frustrating. For instance, I'm a developer for IPython: we made a
> 1.0 release over a month ago, and there's already been a 1.1 release
> since then, but Debian unstable still doesn't have either of these. This
> is not to criticise our packager, who we have a good relationship with,
> but simply to point out that this system is beyond our control. If we
> recommend that people use apt/yum/port/whatever to install IPython,
> they'll get an old package, with bugs that we've already fixed. By
> contrast, we update the packages on PyPI at release time, so users
> installing with pip will always get the current version.
>
> Thomas

Then get involved in the Debian packaging: problem solved!

I try to get involved with Debian packaging. But, to be blunt, it is a slow, frustrating process, and the effort I put in feels utterly disproportionate to the results. We are not going to get many Python programmers involved with the packaging process as it stands. Take the most recent two packages I have worked on:

- python3-sympy: The package is sitting in the team SVN, waiting for someone to review or upload it. I last touched it two months ago to package a new upstream release.

- python-xlrd: My changes were rejected, although the package was extremely out of date, because the package had an individual 'Maintainer' who hadn't been seen for three years. It took another four months (a long time in developer terms) before the wheels turned and someone actually got an up to date version packaged.

I wish I could say this is atypical. But from my limited experience of DPMT, a slow, tricky process is the norm. There are procedural traps (e.g. to make a package you must first file a bug by e-mail, then mark your new package as closing the bug), social traps (annoying a 'maintainer' overprotective of what is ultimately little bit of metadata to turn a Python package into a Debian package), and you may simply fail to catch the interest of a Debian developer--as I seem to have failed with python3-sympy--in which case you're out of luck.

I don't wish to criticise without making suggestions as to how this could be improved:

- Write a 'how to keep your distro packager happy' guide for developers. E.g. many Python developers don't know that distros will move data files to /usr/share, but when you do know, it's easy to write code so that the patch to achieve this is trivial.

- Have a simple way for developers to submit packaging information without having to join Debian teams, file ITP bugs, and all that cruft. Technically, Debian mentors is already something like that, but maintainers mostly ignore it. Which brings me to:

- Put the emphasis on keeping packages up to date and simple, not on 'maintainer rights'. Packaging should not be an art form. If someone uploads a newer version to Debian mentors (or another submission system), the maintainer should get an e-mail. If a package is out of date for more than a few days without a clear reason, people should be prodding the maintainer to ask what's up. If a maintainer is regularly unresponsive, drop the package to team maintainership so that other people can work on it. Put another way, focus on what is best for Debian and for the upstream project being packaged, not what the person who has appointed themself 'maintainer' of that package wants.

- Make it really, really easy to accept changes to packaging. One of the reasons for the meteoric rise of Github is that when someone submits a change that clearly improves things, the repository owner literally just has to click a big green button to accept that. I don't mean DPMT should use Github, but, for instance, if upstream makes a bugfix release 1.4.3, and Debian has 1.4.2, it should be as near as possible one click/one command to grab the new version, update the changelog, commit the change, upload the package, and whatever else needs doing.

I know this won't go down well with everyone here. I contend that if the situation continues as is, Debian packaging will be seen as ever more irrelevant by Python developers. Already, the default assumption is that distro packages will be out of date. The scientific Python community is unhappy with pip & PyPI, but is looking to yet more packaging tools, such as conda, to address this (despite Yaroslav Halchenko's excellent work with NeuroDebian).

Thomas