On Sep 21, 2010, at 03:02 PM, Clint Byrum wrote: >In the java world, they use maven because it handles this for them. >They create a maven spec file that says "I need libX, libY, and >libZ (v1.1)". maven, during the build, goes out and finds libX and >libY's latest versions, then finds the closest match to libZ v1.1. >These are placed in jars in the classpath for the project, and viola >it "just works". Tools like buildout and virtualenv exist to address issues like this. Buildout's goal in life for example is to be able to version control repeatable built environments. It plays quite nicely with Python's own package management system (PyPI, aka Cheeseshop), but poorly with the operating system's package system. This isn't specific to Debian; a Python developer on Fedora would feel the same pain. There have been sessions at probably the last 3 or 4 Pycon sprints to try to address this, but it's difficult. I do think that the work that Tarek and distutils-sig is doing will get us closer, and the more Debian developers that are involved in the process, the better for us. I guess, if it were easy it would be done by now. ;) >But, if you can get the order of the library loading path right, >then this structure solves Robert's original desire: > >/usr/share/pyshared/foo1 >/usr/share/pyshared/foo2 >/usr/share/pyshared/foo => foo2 >/usr/share/pyshared/consumer-of-foo1/lib-packages/foo >=> /usr/share/pyshared/foo1 My sense is that there are two aspects to any possible solution. One is at the OS. Debian needs to support installing multiple versions of a package easily, whether that's by putting the version number in the package name as I think Robert suggests (but I don't like) or some other mechanism. But it shouldn't be doing things like symlinking 'foo' to 'foo2' because only the applications themselves know what versions they are compatible with. This is why buildout has a configuration file that you can use to specific explicitly the library versions you want (and let the dependency management algorithms figure out the rest). It also gives you an isolated environment so that you don't care what the OS has. So something like a configuration file that would ship with the pyfoo app that said "give me foo-1, bar-2.3, and baz-7.0" would set up the interpreter's run-time environment so that those package versions are mapped to "import foo", "import bar", and "import baz". >* What about instances where a dependent-library of consumer-of-foo1 >also wants to 'import foo' but needs foo2? Now I have to make sure >the entire chain works with foo1. How can I do that? IMO, you can't. >* How do I efficiently and reliably prepend that lib-packages >directory only when using consumer-of-foo1 > > >While I think Robert described it as "TERRIBLE" when I suggested >it the other day, the way that pylons does this, I think its at >least simple and understandable. > >For working on the CLI, pylons simply spawns a shell that sets >PYTHONPATH/PYTHONHOME. Likewise, one is required to do so when >running their particular pylons based app as a web application. >This allows you to run easy_install or anything else, inside that >CLI, and the libraries are installed local to the application. I don't like that sub-shell approach much myself, but I've had to do it in a previous life to properly select C++ shared libraries for a compiled application. Of course, this was on Solaris and many years ago. ;). I sympathize with Robert when he calls those approaches "terrible". virtualenv, buildout, and similar are at least less one-off hacky approaches, and the latest version of buildout I think can optionally be told to use system Python installed packages if they match the requested version number. There are folks who think that *every* Python application should essentially have its own isolated environment. I've held the opinion that it's dangerous to use the system Python both for essential system operations (as Debian, Ubuntu, and Gentoo among others do), and also for deployment of applications on those operating systems. I've seen situations in the past that trying to craft a proper environment for a particular app broke the OS. >Its not necessarily ideal, but it shows the great lengths one must >go to to have multiple versions of libraries. Yep. No doubt it's a crappy situation we find ourselves in today. Why not plan on coming to Pycon 2011 and sprinting with the distutils-sig? :) -Barry
Attachment:
signature.asc
Description: PGP signature