Re: PEP 453 affects Debian packaging of Python packages
Hi Paul.
Keep in mind that despite my strong opinions I'm doing my best to be
constructive with what I say below.
W dniu pią, wrz 20, 2013 o 7:52 ,nadawca Paul Tagliamonte
<paultag@debian.org> napisał:
On Fri, Sep 20, 2013 at 10:46:14AM -0700, Thomas Kluyver wrote:
I try to get involved with Debian packaging. But, to be blunt, it
is a slow,
Debian works at a slower pace. We make sound technical decisions, over
all. It's annoying to people who want results and want them now, but
it's something we all have to deal with.
Slow might not be bad *but* arcane and crufty interfaces that are more
reminiscent of the teletype rather than the web should be improved
upon.
Obviously this requires manpower but I want to highlight that it does
put people off from contributing.
frustrating process, and the effort I put in feels utterly
disproportionate to
the results. We are not going to get many Python programmers
involved with the
packaging process as it stands. Take the most recent two packages I
have worked
on:
- python3-sympy: The package is sitting in the team SVN, waiting
for someone to
review or upload it. I last touched it two months ago to package a
new upstream
release.
- python-xlrd: My changes were rejected, although the package was
extremely out
of date, because the package had an individual 'Maintainer' who
hadn't been
seen for three years. It took another four months (a long time in
developer
terms) before the wheels turned and someone actually got an up to
date version
packaged.
I wish I could say this is atypical. But from my limited experience
of DPMT, a
slow, tricky process is the norm. There are procedural traps (e.g.
to make a
package you must first file a bug by e-mail, then mark your new
package as
closing the bug), social traps (annoying a 'maintainer'
overprotective of what
is ultimately little bit of metadata to turn a Python package into
a Debian
package), and you may simply fail to catch the interest of a Debian
developer--as I seem to have failed with python3-sympy--in which
case you're
out of luck.
I don't wish to criticise without making suggestions as to how this
could be
improved:
- Write a 'how to keep your distro packager happy' guide for
developers. E.g.
many Python developers don't know that distros will move data files
to /usr/
share, but when you do know, it's easy to write code so that the
patch to
achieve this is trivial.
- Have a simple way for developers to submit packaging information
without
having to join Debian teams, file ITP bugs, and all that cruft.
Technically,
Debian mentors is already something like that, but maintainers
mostly ignore
it. Which brings me to:
- Put the emphasis on keeping packages up to date and simple, not on
Sorry, no :)
It's not about keeping the libraries up to date, it's about keeping
the
applications up to date.
Sorry but *YES*.
This view might be appropriate for what the world was ten years ago but
now it's not the only "right" way to look at what packages are and who
is consuming them. If Debian has only "stable" packages for "users"
(whatever that means) then people that need packages for other reasons
will simply ignore either Debian or our packages - plain and simple.
I'm not trying to say that we should throw away everything we have and
start over but we *should* recognize that there are other use cases
which we are not serving *at all* by enforcing our current policy.
If a library breaks API because the maintainer wanted another toy
rewrite, we're not going to upload it and break half the archive.
That's
silly.
"toy rewrite" clouds the picture, let's not be the judge here, of what
constitutes a valid reason for breaking the API.
Not all projects have stable API but still have real-world users.
Should those never be in Debian?
Debian should not fit the "ideal spherical cow in vaccum" model. Debian
should support the what-is-really-out-there model.
Hell, we shouldn't even introduce a module unless it has an app using
it.
This is a catch22 problem:
My own experience: I wanted to package my application that I wrote for
Linaro a while ago and all we did was to abandon the ideas as, at that
time, python-celery was not available and the amount of dependencies it
has made the project infeasible. I could have tried packaging them but
then people could reject that with 'apps first' policy.
How many apps out there would be simple to package if we had all of
pypi inside Debian?
We focus on *user stability*, not bleeding edge, yes, even in
Unstable.
Perhaps we can consider exerpimental uploads?
Focusing on user stability *only* misses being "user useful" for
others. Some users suffer because we have old version of a library and
cannot get the "upgrade" required by another application.
Our model assumes that only one version may be installed. This is the
spherical model in vacuum model IMHO. We seem to love special-casing
super-important packages like the kernel, database servers and a few
others, where we support co-installation, but we frown upon the concept
that this is something that is universally needed for all the other
packages.
What might be healthy for the project is to recognize that Debian is
not "maintaining" all of the packages equally. We could provide
packages for all library versions but not provide security fixes
ourselves. We would still have to solve the problem of "picking" the
right library version for a given program but it's certainly in the
"doable" spectrum (especially if we're talking about python)
'maintainer rights'. Packaging should not be an art form. If
someone uploads a
newer version to Debian mentors (or another submission system), the
maintainer
should get an e-mail. If a package is out of date for more than a
few days
without a clear reason, people should be prodding the maintainer to
ask what's
up. If a maintainer is regularly unresponsive, drop the package to
team
maintainership so that other people can work on it. Put another
way, focus on
what is best for Debian and for the upstream project being
packaged, not what
the person who has appointed themself 'maintainer' of that package
wants.
While I generally agree, most of the time the maintainer of a package
knows the details better than anyone else in the project, and can make
better techincal decisions.
I don't think this is true but I have no data to back my claim.
I'd say that for a large majority of packages there *should* be no
single maintainer as that person has no better knowledge than project
developers and the fact that only one person is blessed is an
artificial bottleneck which is impacting users.
For the context of python we could have *no* maintainers and only a
fully automatic packaging process. This goes hand in hand with the
'special subset' for which the Debian project might care more (for
example, because of special security concerns or legacy properties that
are hard to migrate), enough to warrant a human having to "ack" each
automated change as we do now.
- Make it really, really easy to accept changes to packaging. One
of the
reasons for the meteoric rise of Github is that when someone
submits a change
that clearly improves things, the repository owner literally just
has to click
a big green button to accept that. I don't mean DPMT should use
Github, but,
for instance, if upstream makes a bugfix release 1.4.3, and Debian
has 1.4.2,
it should be as near as possible one click/one command to grab the
new version,
update the changelog, commit the change, upload the package, and
whatever else
needs doing.
(check the license, review the diff for changes that may cause issues
or are unsafe, ensure it doesn't break API, ....)
License, yes, because the project is responsible for that.
Everything else - no. If the "maintainer" wants to do that kind of work
I would suggest that he or she joins the developers and just work
upstream.
Why does the Debian project have to second guess everything? Do we
really feel qualified to do that on a large scale?
Maybe if this wasn't happening then we would get more people that
develop the software to give us a hand, as they would be interested in
giving the best possible service to their users. This might also
address some of the scalability issues with the current packaging model.
I know this won't go down well with everyone here. I contend that
if the
situation continues as is, Debian packaging will be seen as ever
more
irrelevant by Python developers. Already, the default assumption is
that distro
We care more about users than developers. Python developers can use
virtualenv and pip on Debian like any other Python development env.
Is this codified anywhere? That Debian project puts the needs of the
users above the needs of users that also happen to develop software?
Your second statement is valid but it only means that packaging for
Debian is ever-more-irrelevant. That hurts everyone as packging
applications is hard without having their libraries packaged (see
catch22 above).
Best regards
Zygmunt Krynicki
Reply to: