Hello, we apologize for the delay in our reply. On Wed, Mar 17, 2010 at 23:50, Bdale Garbee <bdale@gag.com> wrote: > On Sat, 13 Mar 2010 17:24:51 +0100, Sandro Tosi <morph@debian.org> > wrote: >> Package: tech-ctte >> Severity: normal >> >> Hello Technical Committee, >> we'd like you to decide about how the Python interpreter packages >> should be maintained in Debian. > > I've spent several hours since my last message communicating with the > current Python maintainer, reading various mailing list threads and > bug logs, and generally trying to understand the situation as best I > can. Thanks a lot for your investment on this matter. It can be quite time-consuming since, as you noted, things can get pretty emotional. It is sad, though, that this discussion happened in private. We can understand the general reason, but we think having minutes of what was discussed would be interesting to have reported here. Part of the problem is that Matthias is not communicating anymore on public channels, and uses other people as a proxy. To start on solid bases, it would be good if we didn't repeat the same errors from the past. A large part of the perceived issues are directly related to lack of communication, and one of the reasons for our request is Debian cannot live in a situation where the maintainer of a core package doesn't even talk to people who directly depend on his work. Of course, we respect the privacy of those conversations, so we also understand if you're going to decide to keep them private. In any case, if some points arose from those dialogues that you may want us to give examples (or counter-examples), we'd be more than happy to provide them to you. > It bothers me that what you've brought to the TC is a rant about your > frustrations culminating in a request to remove someone else from > their role, rather than a crisper articulation of what's wrong and a > plan that explains how we should move forward. We are sorry if it sounded that way, because this is not the message we wanted to convey. We tried to list the current issues from a technical point of view, and if you want us to elaborate on a specific point, we’ll be happy to provide more input. Overall, our request indeed lacks a complete plan of what we want to implement; because it is not about what should be implemented, but about how it should be discussed, elaborated and coordinated. That said, we understand that you want us to explain what we would propose and implement if we were maintainers for Python, so we will try to elaborate on that. It is not that easy because we have no clear idea of what the current plans of the Python maintainer are. As for removing the current maintainer from his position, it is only one of the possible solutions. We do think that such a core package cannot be effectively maintained by a single person, also given the other points we made in the original email, and that a team should be formed around it. If the current maintainer would like to be part of it, he's very welcome, but at least he should be willing to collaborate; his past actions make us dubious about this, but we are open-minded and hoping for a positive resolution. We were told (again, in private) than one of the proposed members for the team (namely Joss) is a showstopper for Matthias to be part of the said team. We don’t find it a reassuring perspective that Matthias is putting his personal feelings before the best interest for Python in Debian. It was hard for Joss to propose to work together with him given their history, and we’d find it better if everyone involved could make similar efforts. However, if it is required for things to go forward, Joss agreed to step back from the team as a last-resort solution. > In my reading and discussion with > others, there are hints about different agreements at different times > between different people about how to handle various transitions, but > as someone not party to the various discussions over time, I wish I > could find a single, well articulated policy for Python in Debian. > There are more people I want to talk to, and perhaps one or more of > them will make everything clear to me... but I do not want to delay > things further. For a long period of time, the only document that was approaching a Python policy was the complete rewrite initiated by Manoj. The Python policy itself remained incorrect until the upload of python-defaults 2.5.4-4, which brought it mostly on par with the current practice. Let us state it out loud: this was a huge achievement, and we're happy something has moved in the right direction; but please also note that the updated process was not lead by the current Python maintainer, but from (at that time) and external person, that only after his work gained the status of co-maintainer of python-defaults. However there are some points of strong disagreement that remain even in that current policy (not talking about the future), and because of that, maintainers of Python packages don’t all work the same way: * There is no consensus on use of the XS-Python-Version header. The Python maintainer promotes a "current" keyword, which several people think has no semantical value. * There is no consensus on what should be in the Provides header. The policy promotes systematically adding pythonX.Y-foo to this field, which leads to incorrect dependencies being generated. There are some alternate solutions, unfortunately none of which is completely satisfactory. > I realize this may not be what you had in mind when you asked the TC > to "decide about how the Python interpreter packages should be > maintained in Debian", but now that you've opened Pandora's box, I > believe we have a responsibility to understand the underlying > problems that apparently have plagued Python policy for years, so > that whatever decision we take will ensure the most positive outcome > for Python handling in Debian in the future. This is somehow reassuring that the technical committee tries to understand underlying technical issues. Thank you for making this technical and trying to resolve this situation, that's really appreciated. > I'm adding the DPL to this reply because it seems possible that the > only way to achieve this objective might be an in-person Debian > Python Summit meeting, moderated by members of the TC, where we can > work through all the issues and come to consensus. Perhaps we can > resolve our problems by email and/or IRC, but the mere existence of > this petition to the TC and what it implies about communication > disconnects makes me doubt that. Andreas already proposed such a face-to-face summit a few months ago, but ended up abandoning the idea for the moment. This could be the way forward, but it would also be extremely tiring, in an emotional way. Some of the human issues around Python packaging started with a face-to-face meeting that turned out very badly at Debconf 6. > Before I'll be willing to support any Technical Committee action on > this petition, I believe we need a detailed and competent plan > articulated, that explains how we get from where we are today to a > single policy for Python in Debian, and that covers at least: > > 1) our philosophy for handling multiple Python interpreter versions > 2) the supported approach(es) to packaging Python modules > 3) an analysis of the effort involved and who needs to do what > 4) a tentative schedule of milestones to completion, including what > can and should be done before squeeze freeze > 5) explicit commitment by involved parties to do the required work Well, this is one of the points for which the Technical Committee might have to make decisions, since there are strong disagreements in how Python modules should be managed in the future. There are also even stronger disagreements over what parts of it should go into Squeeze. We can try to sum up these disagreements from our point of view if you find it useful. In order to provide some technical background to the petition, and as an example of how we would handle a future similar situation, we'd like to present here what we (as the debian-python community) have done to prepare and support the Python 2.6 transition, to which the current Python maintainer is sadly not participating in any practical standpoint. - We provided impact analysis (packages in need of binNMUs or sourceful uploads) for the 2.6 introduction in the supported python versions and conducted an archive-wide rebuild. - We did the same for the python2.4 removal when Matthias suddenly decided to remove it as well. - We provided patches for debhelper, cdbs, python-central and python-support to support the dist-packages transition. - We asked to schedule more than 150 binNMUs (http://people.debian.org/~dktrkranz/python2.6/python2.6_binNMUs), following their progress, and filing/fixing bugs as they came out or requesting give-backs as needed. - We filed 282 bugs (http://tinyurl.com/Py26Transition) to support the transition, of which 90% is already closed or waiting for an upload. If we consider the python2.4 removal too, this bug count rises to 345. - We made many team uploads, where we uploaded the packages in the Python Modules or Apps team to support the transition. - We prepared NMUs for those bugs, either as direct NMUs or sponsored ones. - We provided help to people who contacted us about how to setup a test environment or how to properly fix their packages for the transition. - We were supported by the Release Team (in the person of Andreas Barth, many thanks!) in the binNMUs part, and we were asked to provide impact analysis for the 2.6 switch as default in unstable (as part of the pre-freeze Release Team plan), showing that with our previous activities we have gained credibility in the eyes of the Release Team. - We asked for a partial-archive wide rebuild to test the impact of setting Python 2.6 as default and filed bugs accordingly. >> transitions to force some controversial, unrelated technical >> changes to be implemented before these transitions happen. > > I think this is a key part of the problem. The Python maintainer does > not seem to believe that these are unrelated technical issues. And > the controversy that does exist seems now to be more fueled by a > combination of emotion and inertia than technical concerns. We need > to get past that, and focus our attentions squarely on a good Python > technical policy and associated implementation plan. I think > everything else will flow fairly easily if we can accomplish that. We don’t think it is enough to know where we are going. We also need to know how to get there and how it integrates in the release process. For example, the dist-packages migration solves a problem most people agreed on (local module installations not going to /usr/local). However, not only was the implementation not discussed (we could have worked on an implementation implying less burden on python modules maintainers), but it was bound to the introduction of python2.6, and it delayed the migration to this version, which otherwise would have been a matter of weeks (unlike 2.4→2.5, the 2.5→2.6 upgrade doesn't require any changes to existing sources). Furthermore, the strategy for handling Python modules can have a strong impact on the release schedule. For example, if we want to change for the better the way modules are handled in the future, we need the following: * expose possible strategies - we are barely here yet; * reach consensus on the best solution; * implement whatever needs to be implemented; * make packages transition - depending on the solution, this can take a lot of time, some of the solutions requiring sourceful changes to all Python packages. It was completely unrealistic to be able to do that before the squeeze release, and it became even more unrealistic over time when even the first item was not dealt with. Yet Matthias wants these changes implemented before python2.6 to be introduced in unstable. This is now putting the release schedule at risk, since we don’t have much time before the freeze (and the Release Team already made us a big gift granting us 1 months for the transition in the pre-freeze plan), and it is not reasonable to release without Python 2.6 being the default - at least, our users wouldn't understand. This is the key point of our request to the Committee. We have no problem with packaging changes, but at the following conditions: * they must be discussed with other maintainers of Python modules; * they must make things better for our users; * they should not have a too big impact on the rest of the distribution. Therefore, we want to maintain the Python packages and implement such changes, but only with consensus from the community and using open methods. In the specific case that is being discussed currently, we think the packaging changes - if they are needed - need to be postponed after the python2.6 transition. We can start to discuss and implement things before the freeze, but we think it’s too late to start a transition now. Said otherwise, the "python2.6 as default" and "change everything in the way to package Python modules" transitions need to be handled separately and sequentially. This is also the usual way the Release Team likes maintainers to work. Furthermore, the specific changes that have been proposed are high risk, and several questions need to be answered before the transition can be considered. - How will the new helper be used? As a drop-in replacement or something different? - Will it replace any of the existing helpers? If yes, which and how? - Will the maintainers using the removed helpers be forced to use the new one? - How will we ensure the correct behavior of the new helper in all the archive? - Are there any functional regressions? - And more importantly than anything else, where is the policy document that specifies the implementation? These are the questions that would have arisen automatically, had the proposal been discussed openly within the Debian/Python community. And we would have answers instead of being in the blue. Kind regards. Luca, Josselin, Sandro, Bernd
Attachment:
signature.asc
Description: PGP signature