On May 23, 2010, at 03:58 PM, Piotr Ożarowski wrote: >[Barry Warsaw, 2010-05-22] >> So, how can we make sure that doesn't happen? IOW, how can I begin to >> experiment with a Python 2.7 transition in a way that will benefit Debian as >> well?' > >Simply avoid doing transitions with significant changes in either upstream >code¹ or in interpreter packaging² as otherwise the transition will be >made by few overloaded developers who have to update lots of packages really >urgently (just few days ago I noticed yet another "fix" made by Python >core developer this time) or by people who don't use³ Python at all. > >[¹] f.e. changes in API, absolute imports in 2.7 would be one of those, > right now I don't see such changes in 2.7, but I didn't play much > with it yet Contrary to what PEP 328 implies, absolute imports are *not* the default in Python 2.7. Default import semantics will only change in Python 3. >[²] f.e. --prefix or dist- vs. site-packages change I don't foresee any such additional packaging changes of this type for Python 2.7. >[³] without seeing the bigger picture (i.e. not being a long term > maintainer for given package), it's really easy to make a mistake, > even if a package builds fine on all architectures, passes all tests > (with 100% test coverage), is lintian and piuparts clean, module imports > fine, works fine... you still can end with bugs like LP:418280 (and I > can easily imagine breaking lots of other packages by such bugs). > Note that I assume Ubuntu maintainer *did* check all these things > before uploading. Of course, that tells me that there's a bigger problem here. :) IIUC, you're saying the Ubuntu maintainer probably did all the right things and yet stuff was still broken. If that's the case, then it seems like there's no other way to find these breakages short of just trying to make the transition happen and waiting for bug reports. LP #418280 does seem like a process bug: a package that previously was unbundled got added to the stdlib, making the separate package obsolete. The one I can think of for Python 2.7 would be argparse. >> experimental's got >> Python 2.6.5 right now, but once that lands in squeeze does it make sense to >> get Python 2.7 into experimental? > >testing, unstable and experimental have Python 2.6.5 right now, the only >difference is experimental has 2.6 set as default since few days. 2.7 is >in experimental as well, it's not marked as supported, though. Adding >2.7 to the list of supported Python versions will make it a lot easier >to build packages with 2.7 support (f.e. in experimental enabled >pbuilder) and test it in unstable (with 2.7 installed from experimental) What do we need to do to make Python 2.7 supported in experimental? >> >> https://wiki.ubuntu.com/MaverickMeerkat/TechnicalOverview/Python >> > >> >* how many Ubuntu developers at least read PEP 3147? >> >> Probably only a handful. > >sorry, but I don't think it's enough to do the python3.2 transition in >few months (please note that we (I?) were told that that f.e. pyshared >related changes in python-central were tested really well by Ubuntu >developers) While I'd love to be bold and get Python 3.2 in Ubuntu 10.10, if we're time constrained I personally think Python 2.7 is a higher priority, because the latter will have been released upstream by then, but the former will not. It will already be problematic to put 3.2 in Maverick because we'll have to get pre-approval for SRUs to match final release. Note that doesn't mean we still shouldn't begin to do the testing and preparation for Python 3.2. Making it a goal doesn't make it a certainty (for this cycle anyway :). >> I'd guess that only a handful of upstream Python >> core devs have read it too though. ;) > >I think most upstream Python core devs do not try to use two (or more) >different Python versions with hundreds of 3rd party (sometimes really >old) modules and applications, so they probably do not care much about >issues that PEP3147 tries to solve. I don't know. I think many upstreams try to test with multiple versions of Python, but I don't know if they do it simultaneously, how much they rely on the platform's Pythons, etc. I'm also not quite sure what the point of the question is. ;) But you're probably right that most upstreams don't care too much about the central problem that PEP 3147 solves (though *lots* of people seem to want the directory de-cluttering aspects of it). >> OTOH, I think most Python developers, >> and Python packages, will not be affected by PEP 3147 since most don't care >> about where their pyc files live. > >Python developers (including 3rd party module authors) - sure (if they >have to care about it, it's a bug in Python itself, IMHO) > >Python packages - I don't agree with you here. >Before I will ask you my next question (and point to issues with >dist-packages change), please let me know how do you want to share .py >files between two different Python versions (including exceptions where >.py file cannot be shared, f.e. due to __future__ imports). I agree we definitely have to figure this out. You've made some proposals which make sense to me, though I want to keep thinking about it to see if we can come up with a better idea. (But if we think we can solve it upstream, we better do it before 3.2 beta comes out!) >> Some will of course and we will have to fix >> them or ask their authors to fix them. Still, I suspect that the intersection >> of packages that care *and* are ported to Python 3 is probably ridiculously >> small right now. ;) That would change of course if we backport to Python 2. > >If Ubuntu will introduce something that Debian will never support or >will choose to do it differently, there will be problems sooner or >later. The fact that you will start with small number of packages >doesn't really matter. Avoiding that is why we're discussing things here! >> >* how many of them will work on huge changes that packages need in order >> > to work with PEP3147? >> >> I don't know. What exactly needs to change? > >f.e. helper tools are not ready for it; we didn't agree how to share .py >files on hard disk yet; some packages started adding 3.x support in the >binary package which provides 2.X module (and it looks like python3 >maintainer wants to separate them)... is that enough for a start? (I'm >afraid the list will be much longer) It's a good start, but I would really like to start enumerating these in a place that isn't as volatile as an email thread. Can you suggest a good place on the Debian wiki to do this? If not, I will make a place for it on wiki.python.org or wiki.ubuntu.com. >> Not the packages themselves, but >> the packaging? How much of that can be automated? > >I cannot tell you what can be automated if I don't know which path will >be chosen (I'm still waiting for Matthias' proposals and discussion on >this mailing list) Let's see what we can come up with while we wait for that. >> >* will Ubuntu force Debian to use these changes (you will *have to* >> > introduce them in 10.10 if you want to use python3.2 on desktop)? >> >> I hope not *force*. I hope we can do a lot of the work and make it available >> to Debian. > >By "forcing" I mean introducing changes in Ubuntu and uploading to >Debian later, without our influence on design decisions or possibility >to change something and with months of delays between Ubuntu and Debian >uploads (see f.e. python-central NMU saga and reasons why I finally >decided to switch to python-support) So, let's not do that <wink>. We're discussing issues here so that we can reach consensus between all of us who care about and love Python, and want the best distributions of it that we can provide. >> >FYI: I've been there, I tried to fix 2.6 related bugs in Ubuntu without >> >possibility to test them on Debian, it's not fun at all and I will not >> >do it again (since I got not nice comments and delays in return) >> >> Well, if we understand now the types of problems that can come up, then >> hopefully we can coordinate fixes for them in a much more positive way. I >> still don't have a good understanding of what - outside of Python 2.7 >> compatibility itself - needs to change. > >Python 2.6.4 -> 2.6.5 upgrade broke one of my packages in Ubuntu, I >suspect the diff between Python 2.6.5 and 2.7 is a little bit bigger >and thus there are more potential problems. I don't care about which 2.x >version Maverick will ship anymore (I already left #ubuntu-motu). I just >hope it will not introduce packaging changes that will break "my" >packages or "my" packages will not be changed by people who don't use >them. Upstream certainly tries not to cause backward incompatibility between point releases, but as you know, developing software is a dirty, ugly, inexact business. :) I don't think there's any way to 100% avoid breaking stuff in anything you do, especially with something as critical, complex, and big as Python, with as many developers in the pot of stew. The best you can do is to be diligent, thoughtful, and thorough, backed up with testing, openness and collaboration. So what broke for you in 2.6.4 -> 2.6.5? Please do keep in mind that putting Python 2.7 in Maverick is also a goal. It may ultimately prove infeasible, but I do hope that we have put a lot of thought into it and have a lot more data on the other end. When we do transition (and it is inevitable at some point) we'll be that much farther along. -Barry
Attachment:
signature.asc
Description: PGP signature