At 11:36 AM 11/24/2005 -0800, Robert Kern wrote:
Phillip J. Eby wrote: > Note, by the way, that those two things are the only essentials here, as > best I can tell, and I've already stated my willingness to change *how* > those two things get accomplished. For clarity, I will repeat yet again, > in yet another way: > > 1. Egg-based projects need to install their published metadata, in a> well-known location relative to the installation location of their code, so> that it can be found by searching sys.path, so that it and other projects > can locate the metadata for currently-importable projects, *without* > needing to first import the project's code. > > 2. Egg-based projects need to be able to identify whether another Python > project package is installed and what version it is, without requiring > modification to that other project's code or needing to import it. (And > this is independent of whether the depended-on project was packaged as an > egg by its author.) > > As far as I'm aware, those are the irreducible technical minimum > requirements for making an egg-based project work. *How* these > requirements are met is quite flexible, as there are already three working > layouts that achieve this. As I said before, I'm quite willing to > implement a fourth. But nobody has been proposing anything that meets > these requirements, because they're too busy trying to prove the > requirements don't exist or are somehow not real. [Note: I am a happy Debian user, though not a DD. I am also one of the developers of a Debian-packaged Python package, and we're considering using pkg_resources to implement certain new features. I swear, this is like watching two parents fight. Anyways...] I think one of the sticking points with the Debian developers has been that the .egg-info metadata is being put into /usr/lib/... when according to Debian policy and general UNIX lore, such should be placed somewhere in /usr/share/.... Would it be possible to treat /usr/share/pythonX.Y-egginfo/ as a proxy for /usr/lib/pythonX.Y/site-packages/ when searching for .egg-info directories?
The sticking point is the relative relationship to a sys.path directory (line 2 of requirement 1, above). Since a program such as Trac can legitimately manipulate sys.path to e.g. add plugin directories, we need to be able to do some reasonable transformation to *every* sys.path entry, preferably without introducing some kind of distro-specific configuration to match special directories and redirect them. We really want to treat every sys.path entry as a valid place to find a project's metadata, since each is a valid place for the actual packages to live.
There is of course also a performance issue; currently we need only scan each sys.path directory at most once in order to locate all packages and their metadata. (.egg directories don't need to be scanned unless looking for a dependency that's not already on sys.path.) The scan of non-.egg directories identifies any .egg-info directories there. So, for a cost of zero or one listdir() for each sys.path entry, we can find the locations of all the discoverable metadata.
However, if the issue here is where the data files actually live, couldn't Debian just put the egg info wherever it wants, and symlink to it from site-packages? That would already work with the current runtime, although it seems to me like it's just making more work for the Debian packagers, for no benefit that I comprehend. (Especially since in the case of non-setuptools packages, it might be just an empty file anyway.)
Note that Python packages have always been able to include data files inside their package directories, and in fact the *single most popular* distutils modification made by published Python packages is to *not* install data under /usr/share and instead put those files in the packages' directories, where the package can find it without needing to know the platform-specific or distro-local place for data. This is why setuptools pioneered an easier way to add package data almost 2 years ago, and that way was officially backported into the distutils in Python 2.4. See:
http://python.org/doc/2.4.1/dist/node11.html
Anyway, symlinks would work, but it seems to me rather inconsistent to push
project-level data to /usr/share while leaving module-level data in
/usr/lib. And pushing either one of the two to /usr/share seems like
wasted effort on a non-problem to me. In neither case is the data going to
be changed; it can be just as read-only as the code is.
(But just because I don't think it's useful, I'm not going to act like it's not a real problem for the Debian team, or insist that putting files in /usr/share is undesirable. If it's desirable to them, it's desirable to them. The only relevant question is how can we both get our desires met.)