At 07:54 AM 11/25/2005 +1100, David Arnold wrote:
From a Python-centric viewpoint, Debian's (and RedHat, Gentoo, Solaris, etc) packaging mechanism, however great, covers only one of the many possible platforms that an application might need to support. From a Debian-centric viewpoint, dependency management is a solved problem, with a decade of experience behind it, and proposing that Python applications should have their dependencies managed separately is blatantly foolhardy.
I prefer a slightly different spin:Python developers would *love* to have Debian manage their packages, they would simply like to be able to verify at runtime that the management has in fact been done. It's not that we don't trust you, it's just that we're paranoid. :) We'd like for Debian to advertise to our packages, precisely what versions of which of our dependencies are installed. We'd also like for Debian to include the metadata we provided with our packages, when it installs them. And we'd like all this to cleanly work with any locally-installed non-Debian eggs that might be in the mix, since we need to do development, beta testing, etc.
I'm not convinced that's the case (I'm not sure how reasonable it is to
automatically convert 'require("FooBar>=1.2")' into the equivalent
Debian package names and versions, but my expectation is that it's no
more difficult than it currently is to package a "product" for Debian.
The 'FooBar' name generally refers to a well-known name registered with PyPI, and both the name and version come from metadata that is already supplied by setup.py for all distributions packaged with the distutils today. So you actually have this information *already*, for any distutils-based distribution you package with Debian.
For the Python developer's purposes (i.e., verifying whether something is there), you do not *need* to match this up with Debian package naming, however. You only need to do that when you are packaging something that uses such dependencies, and as you say, it's probably not any harder than the dependency mapping you do now. (Except for the part where egg-based projects tend to have more dependencies simply because they *can*, encouraging reuse instead of reinvention.)
Anyway, for advertising the presence of dependencies, we only need a standardized way for distributors to say, "FooBar 1.2 is installed here". If a developer's project needs FooBar 1.2, he's not really worried about what Debian package provides it, and the egg runtime isn't in the business of asking system packagers to install things. It just "properly complains" (as one person put it) about the absence of things that should be there.
Finally, this is ignoring other possible uses of egg metadata, which might end up being far more problematic?
I don't see why they would be any more problematic than any other kind of data that comes with Python packages. Again, we just need to be able to read this information at runtime to discover services, plugins, etc. without requiring user configuration besides just installing things.
Here's a motivating example of what this kind of data can be used for. Suppose I said, "okay, I don't see how we can satisfy the Debian requirements cleanly, so suppose I make setuptools allow plugins to redefine how the metadata gets found?"
I would then define an entry point group, let's say "setuptools.dependency_resolvers", and add some code to query these entry points when scanning for egg metadata. Debian could then create a separate Python package, and include in its metadata an entry point for this entry point group, which would tell setuptools where to import the plugin code from. Debian would then make sure this package and its metadata are discoverable to setuptools, and setuptools would then automatically do the right thing on any Debian installation, even if the user was running with a new, locally-installed version of setuptools.
Of course, this scenario wouldn't actually work due to the bootstrapping issue. I just mention it so you can see an example of why packages want to be able to discover metadata besides the mere presence of a dependency. In this example, the hypothetical Debian-written plugin need not be known about by the provider of the plugin point, i.e. setuptools.
Anyway, the point is that we simply need a way to link that kind of data to sys.path entries, so we can be sure that the things we need are there, and not shadowed by things that are before them on sys.path.