[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#573745: Please decide on Python interpreter packages maintainership

we apologize for the delay in our reply.

On Wed, Mar 17, 2010 at 23:50, Bdale Garbee <bdale@gag.com> wrote:
> On Sat, 13 Mar 2010 17:24:51 +0100, Sandro Tosi <morph@debian.org>
> wrote:
>> Package: tech-ctte
>> Severity: normal
>> Hello Technical Committee,
>> we'd like you to decide about how the Python interpreter packages
>> should be maintained in Debian.
> I've spent several hours since my last message communicating with the
> current Python maintainer, reading various mailing list threads and
> bug logs, and generally trying to understand the situation as best I
> can.

Thanks a lot for your investment on this matter. It can be quite
time-consuming since, as you noted, things can get pretty emotional.

It is sad, though, that this discussion happened in private. We can
understand the general reason, but we think having minutes of what was
discussed would be interesting to have reported here. Part of the
problem is that Matthias is not communicating anymore on public
channels, and uses other people as a proxy. To start on solid bases,
it would be good if we didn't repeat the same errors from the past. A
large part of the perceived issues are directly related to lack of
communication, and one of the reasons for our request is Debian cannot
live in a situation where the maintainer of a core package doesn't even
talk to people who directly depend on his work.

Of course, we respect the privacy of those conversations, so we
also understand if you're going to decide to keep them
private. In any case, if some points arose from those dialogues
that you may want us to give examples (or counter-examples), we'd
be more than happy to provide them to you.

> It bothers me that what you've brought to the TC is a rant about your
> frustrations culminating in a request to remove someone else from
> their role, rather than a crisper articulation of what's wrong and a
> plan that explains how we should move forward.  

We are sorry if it sounded that way, because this is not the message we
wanted to convey. We tried to list the current issues from a technical
point of view, and if you want us to elaborate on a specific point,
we’ll be happy to provide more input. Overall, our request indeed lacks
a complete plan of what we want to implement; because it is not about
what should be implemented, but about how it should be discussed,
elaborated and coordinated. That said, we understand that you want us
to explain what we would propose and implement if we were maintainers
for Python, so we will try to elaborate on that. It is not that easy
because we have no clear idea of what the current plans of the Python
maintainer are.

As for removing the current maintainer from his position, it is only one
of the possible solutions. We do think that such a core package cannot
be effectively maintained by a single person, also given the other
points we made in the original email, and that a team should be formed
around it. If the current maintainer would like to be part of it, he's
very welcome, but at least he should be willing to collaborate; his
past actions make us dubious about this, but we are open-minded and
hoping for a positive resolution.

We were told (again, in private) than one of the proposed members for
the team (namely Joss) is a showstopper for Matthias to be part of the
said team. We don’t find it a reassuring perspective that Matthias is
putting his personal feelings before the best interest for Python in
Debian. It was hard for Joss to propose to work together with him given
their history, and we’d find it better if everyone involved could make
similar efforts. However, if it is required for things to go forward,
Joss agreed to step back from the team as a last-resort solution.

> In my reading and discussion with
> others, there are hints about different agreements at different times
> between different people about how to handle various transitions, but
> as someone not party to the various discussions over time, I wish I
> could find a single, well articulated policy for Python in Debian.
> There are more people I want to talk to, and perhaps one or more of
> them will make everything clear to me... but I do not want to delay
> things further.

For a long period of time, the only document that was approaching a
Python policy was the complete rewrite initiated by Manoj. The Python
policy itself remained incorrect until the upload of python-defaults
2.5.4-4, which brought it mostly on par with the current practice.

Let us state it out loud: this was a huge achievement, and we're
happy something has moved in the right direction; but please also note
that the updated process was not lead by the current Python maintainer,
but from (at that time) and external person, that only after his work
gained the status of co-maintainer of python-defaults.

However there are some points of strong disagreement that remain even
in that current policy (not talking about the future), and because of
that, maintainers of Python packages don’t all work the same way:

* There is no consensus on use of the XS-Python-Version header. The
  Python maintainer promotes a "current" keyword, which several people
  think has no semantical value.
* There is no consensus on what should be in the Provides header. The
  policy promotes systematically adding pythonX.Y-foo to this field,
  which leads to incorrect dependencies being generated. There are some
  alternate solutions, unfortunately none of which is completely

> I realize this may not be what you had in mind when you asked the TC
> to "decide about how the Python interpreter packages should be
> maintained in Debian", but now that you've opened Pandora's box, I
> believe we have a responsibility to understand the underlying
> problems that apparently have plagued Python policy for years, so
> that whatever decision we take will ensure the most positive outcome
> for Python handling in Debian in the future.

This is somehow reassuring that the technical committee tries to
understand underlying technical issues. Thank you for making this
technical and trying to resolve this situation, that's really

> I'm adding the DPL to this reply because it seems possible that the
> only way to achieve this objective might be an in-person Debian
> Python Summit meeting, moderated by members of the TC, where we can
> work through all the issues and come to consensus.  Perhaps we can
> resolve our problems by email and/or IRC, but the mere existence of
> this petition to the TC and what it implies about communication
> disconnects makes me doubt that.

Andreas already proposed such a face-to-face summit a few months ago,
but ended up abandoning the idea for the moment. This could be the way
forward, but it would also be extremely tiring, in an emotional way.
Some of the human issues around Python packaging started with a
face-to-face meeting that turned out very badly at Debconf 6.

> Before I'll be willing to support any Technical Committee action on
> this petition, I believe we need a detailed and competent plan
> articulated, that explains how we get from where we are today to a
> single policy for Python in Debian, and that covers at least:
> 1) our philosophy for handling multiple Python interpreter versions
> 2) the supported approach(es) to packaging Python modules
> 3) an analysis of the effort involved and who needs to do what
> 4) a tentative schedule of milestones to completion, including what
> can and should be done before squeeze freeze
> 5) explicit commitment by involved parties to do the required work

Well, this is one of the points for which the Technical Committee might
have to make decisions, since there are strong disagreements in how
Python modules should be managed in the future. There are also even
stronger disagreements over what parts of it should go into Squeeze. We
can try to sum up these disagreements from our point of view if you
find it useful.

In order to provide some technical background to the petition,
and as an example of how we would handle a future similar
situation, we'd like to present here what we (as the
debian-python community) have done to prepare and support the
Python 2.6 transition, to which the current Python maintainer is sadly
not participating in any practical standpoint.

- We provided impact analysis (packages in need of binNMUs or sourceful
  uploads) for the 2.6 introduction in the supported python versions and
  conducted an archive-wide rebuild.
- We did the same for the python2.4 removal when Matthias suddenly
  decided to remove it as well.
- We provided patches for debhelper, cdbs, python-central and
  python-support to support the dist-packages transition.
- We asked to schedule more than 150 binNMUs
  following their progress, and filing/fixing bugs as they came out or
  requesting give-backs as needed.
- We filed 282 bugs (http://tinyurl.com/Py26Transition) to support the
  transition, of which 90% is already closed or waiting for an upload.
  If we consider the python2.4 removal too, this bug count rises to 345.
- We made many team uploads, where we uploaded the packages in the
  Python Modules or Apps team to support the transition.
- We prepared NMUs for those bugs, either as direct NMUs or sponsored
- We provided help to people who contacted us about how to setup a
  test environment or how to properly fix their packages for the
- We were supported by the Release Team (in the person of Andreas
  Barth, many thanks!) in the binNMUs part, and we were asked to
  provide impact analysis for the 2.6 switch as default in
  unstable (as part of the pre-freeze Release Team plan), showing
  that with our previous activities we have gained credibility in
  the eyes of the Release Team.
- We asked for a partial-archive wide rebuild to test the impact of
  setting Python 2.6 as default and filed bugs accordingly.

>> transitions to force some controversial, unrelated technical
>> changes to be implemented before these transitions happen.
> I think this is a key part of the problem.  The Python maintainer does
> not seem to believe that these are unrelated technical issues.  And
> the controversy that does exist seems now to be more fueled by a
> combination of emotion and inertia than technical concerns.  We need
> to get past that, and focus our attentions squarely on a good Python
> technical policy and associated implementation plan.  I think
> everything else will flow fairly easily if we can accomplish that.

We don’t think it is enough to know where we are going. We also need to
know how to get there and how it integrates in the release process. 

For example, the dist-packages migration solves a problem most people
agreed on (local module installations not going to /usr/local).
However, not only was the implementation not discussed (we could have
worked on an implementation implying less burden on python modules
maintainers), but it was bound to the introduction of python2.6, and it
delayed the migration to this version, which otherwise would have been
a matter of weeks (unlike 2.4→2.5, the 2.5→2.6 upgrade doesn't require
any changes to existing sources).

Furthermore, the strategy for handling Python modules can have a strong
impact on the release schedule. For example, if we want to change for
the better the way modules are handled in the future, we need the
 * expose possible strategies - we are barely here yet;
 * reach consensus on the best solution;
 * implement whatever needs to be implemented;
 * make packages transition - depending on the solution, this can take
   a lot of time, some of the solutions requiring sourceful changes to
   all Python packages.

It was completely unrealistic to be able to do that before the squeeze
release, and it became even more unrealistic over time when even the
first item was not dealt with.

Yet Matthias wants these changes implemented before python2.6 to
be introduced in unstable. This is now putting the release
schedule at risk, since we don’t have much time before the
freeze (and the Release Team already made us a big gift granting
us 1 months for the transition in the pre-freeze plan), and it is
not reasonable to release without Python 2.6 being the default -
at least, our users wouldn't understand.

This is the key point of our request to the Committee. We have no
problem with packaging changes, but at the following conditions:
 * they must be discussed with other maintainers of Python modules;
 * they must make things better for our users;
 * they should not have a too big impact on the rest of the
   distribution. Therefore, we want to maintain the Python packages and
   implement such changes, but only with consensus from the community
   and using open methods.

In the specific case that is being discussed currently, we think the
packaging changes - if they are needed - need to be postponed after the
python2.6 transition. We can start to discuss and implement things
before the freeze, but we think it’s too late to start a transition
now. Said otherwise, the "python2.6 as default" and "change everything
in the way to package Python modules" transitions need to be handled
separately and sequentially. This is also the usual way the Release
Team likes maintainers to work.

Furthermore, the specific changes that have been proposed are high
risk, and several questions need to be answered before the transition
can be considered.
- How will the new helper be used? As a drop-in replacement or
  something different?
- Will it replace any of the existing helpers? If yes, which and how?
- Will the maintainers using the removed helpers be forced to use the
  new one?
- How will we ensure the correct behavior of the new helper in all the
- Are there any functional regressions?
- And more importantly than anything else, where is the policy
  document that specifies the implementation?

These are the questions that would have arisen automatically, had the
proposal been discussed openly within the Debian/Python community. And
we would have answers instead of being in the blue.

Kind regards.
Luca, Josselin, Sandro, Bernd

Attachment: signature.asc
Description: PGP signature

Reply to: