Re: [Distutils] formencode as .egg in Debian ??
At 09:04 PM 11/25/2005 +1100, David Arnold wrote:
-->"Phillip" == Phillip J Eby <pje@telecommunity.com> writes:
Phillip> I would like to clarify the phrase "shipped as an egg",
Phillip> though. To me, that would mean that the developer is
Phillip> distributing a binary .egg file, and I'm assuming that Debian
Phillip> is primarily interested in *source* packages, being a Free
Phillip> Software distribution.
Debian ships both source packages and binary packages. It's my
impression (although I could be wrong) that most users use only the
binary packages.
I was saying I assume Debian packages (source or binary) are built from
upstream source packages, but you were asking about upstream packages
"shipped as an egg". .egg files do not include the full source of a
disutils package, except perhaps the .py files. C source would not be
included in a distributed .egg. Anyway, I assume that Debian Python
packages are currently built from upstream source, not an upstream binary,
and .egg files would be considered a binary.
Assume a developer grabs some setuptools-using Python app, and tries to
run it on a Debian system. It will look in sys.path for its
dependencies. If it doesn't find the egg info, as I understand it, the
app will then go out and grab those dependencies via PyPI until it's
able to run.
No, the default behavior will be to throw a DependencyNotFound exception
describing the problem. A particular application can in principle catch it
and do something about it, or pass in an installer hook that will be tried
before throwing the exception, but that's up to the application. The
setuptools runtime by default is just going to give an exception. Indeed,
if it's a required dependency (as opposed to a dependency needed for an
optional feature of the application), then you're not going to be able to
start the application at all; you'll just get the DependencyNotFound
exception currently.
But, if compatible versions of those dependencies are already installed
as Debian packages *without* egg metadata, will these be ignored?
That's correct; the wrapper scripts presently have no way to tell such
packages exist, and no reliable uniform way to check their version. One
possible feature that has been discussed is allowing package authors to
specify fallback hooks in their application to try to verify the presence
of a package by importing it and testing for the existence of certain
symbols. A couple of developers have suggested that for some dependencies,
they would be willing to specify such fallbacks in order to take advantage
of non-egg system packages. At this point, the means by which this could
happen is incompletely specified, and it would be tricky to implement
because it means importing code from partially processed dependencies in
order to process *their* dependencies, which means the dependency
resolution process can't be safely unwound and retried.
The other thought I'd had about this (aside from adding .egg-info files to
every Python package), is to provide an entry point group that could be
registered by a module provided by a system packager that would use the
packager's native API to test for the presence of a dependency, and supply
a mock Distribution object with the metadata. Of course, this hypothetical
packager API would have to be able to map PyPI names and versions to native
package names and versions and obtain the metadata, so I'm not sure that in
the general case such a thing would be any more useful. (Aside from the
fact that such a feature would allow you to put the metadata in /usr/share
without adding that directory to sys.path.)
Even if it was possible for Debian to extend this downloading mechanism
so that it looked for dependencies via apt-get before trying to install
from the raw source or egg or whatever,
Again, this is not the default behavior. It's the "easy_install" tool that
does such downloading and installing, or "setup.py install". Downloading
and running things from the internet by default didn't seem to me like a
safe thing to do. :) There are various "setup.py" commands that will do
it (such as "develop" and "test"), but by running a setup.py you're
trusting the author to install arbitrary stuff on your machine anyway. The
installed application, unless it provides some explicit user functionality
to say, download plugins, will not have this download-and-install behavior.
I'm assuming that a Debian user will not be using "easy_install" except to
get and run bleeding edge stuff, and that in that case they will be
directing it to install in /usr/local or a personal directory.
it would usually be the case
that the user running the newly downloaded Python application would not
have permission to install system packages.
Ultimately, I suppose this isn't a major problem -- if a user (or
developer) runs such an application, they might end up pulling down and
installing a bunch of dependencies in their home directory (that's where
they'd go by default, right?)
No. The default is determined by the standard distutils configuration file
chain. If an install location isn't specified in the user's
~/.pydistutils.cfg, the systemwide distutils.cfg will be checked, and
finally the default will be /usr/lib/.../site-packages. Debian, by the
way, apparently ships with this default unchanged; it should presumably
point to /usr/local/.../site-packages instead, so that locally-installed
packages would go to the right place. Or you could set a default location
such that easy_install will install to a home-relative location by default,
and include that directory in your .pth processing (as is done by Mac OS
Python with the ~/Library/.../site-packages directory).
But once the app is packaged, both it and
its dependencies will be available for all users, so it'll just be early
adopters who will encounter this.
Correct, and only if they are using easy_install or a setup.py to install
the package in the first place. And a Debian user using Debian packages
should have no reason to touch easy_install, unless they lack permission to
install packages.
Reply to: