Re: [Distutils] formencode as .egg in Debian ??
At 10:29 AM 11/25/2005 +0000, Donovan Baarda wrote:
On Fri, 2005-11-25 at 01:33 -0500, Phillip J. Eby wrote:
[... long informative explanation of egg...]
So it sounds like egg duplicates much of the functionality of the Debian
Package manager, but also extends it slightly in python-specific ways.
I can under stand the Debian packagers asking themselves; do we just
wrap an egg in a Debian package and accept the duplicated functionality,
It's not duplicated functionality in normal operation. In the case where a
user decides to use easy_install to install something instead of using a
Debian package (e.g. because it's not available as one yet), it simply
allows them to *also* use Debian packages to satisfy dependencies. If the
user wants to pull an up-to-the-minute TurboGears, which uses a mix of
other bleeding-edge stuff and stable stuff, it'd be nice for them to be
able to use their system packages to satisfy those more stable dependencies.
In the case where the user is *not* using easy_install, then all
dependencies will be met by system packages, and the only "duplication" is
that the paranoid setuptools integrity-check wrappers will want to be
assured that yes, everything is still there.
The biggest risk of just wrapping the duplicated functionality is; are
there incompatibilities or policy violations in the egg way of doing
things?
In particular, will an egg wrapped inside a Debian package magically
install other bits of software not from Debian packages? Will it install
them in the correct places?
Not unless the application was specifically written to do this. For
example, I believe Trac has a web interface that allows you to upload
plugin eggs to the Trac application, which will go in that specific Trac
installation's plugin directory.
However, there is *no* automatic download-and-install for eggs *except* by
running setup.py commands or by using easy_install. Any
download-and-install features provided by a particular application or
library are that author's responsibility. And I deliberately didn't make
it *too* easy to implement that functionality, because I believe people
should think carefully about the ramifications of download-and-install
without the user explicitly running setup.py or easy_install. In
particular, most applications needing such functionality should be
downloading into an application-specific plugin directory (which they then
add to their sys.path at runtime), and definitely not to system locations.
If TurboGears depends on an "egg'ed" ElementTree, what happens if a
system has ElementTree installed as a non-egg package? Does installing
TurboGears as an egg inside a Debian package require also installing
another ElementTree as an egg inside a Debian package? Or worse, will it
automatically download and install an egg'ed ElementTree not from a
Debian package? Will you end up with two ElementTree's installed, one
egg'ed and one not?
If you are running easy_install or setup.py to install TurboGears, and an
egged ElementTree is not available, one is going to get installed for you
-- because in that scenario you are asking for installation and are
presumably doing it to /usr/local or to ~/somewhere. If you are merely
*using* TurboGears and an egged version of a dependency is not available,
you will get a DistributionNotFound exception at startup or during runtime,
depending on how the author defined the dependency and whether it was an
optional one. (Optional dependencies allow the package author to define
optional features for which additional dependencies are required. Usually
this is in the form of self-registered entry points so that the
dependencies are checked just before trying to import and invoke the code
that implements the feature. The application can then trap any
DistributionNotFound or VersionConflict errors and continue without the
feature.)
For more info about optional features, see:
http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras-optional-features-with-their-own-dependencies
Debian has versioned dependencies. If a package depends on a particular
version of another package that doesn't exist yet, you can't install it;
full stop. If you want to package something that depends on a SVN
revision of some dependency, then you need to package that SVN version.
Right; the point here is that if somebody is going *outside* of Debian
packages and explicitly using setup.py or easy_install to install it, then
it would be nice for them to *still* be able to use system packages to meet
some of the dependencies, rather than having to re-download, re-build, and
re-install new copies of the stuff they already have. This isn't about
changing Debian to support bleeding edge things, it's about allowing people
doing bleeding edge things to not need to duplicate what Debian already
provides.
I think they a both important. If I was using eggs, I'm not sure I'd
want them to be packaged as anything other than an egg until I was ready
to release, and then I'd want it **not** packaged as an egg, as a "proof
of it's release readiness".
Well, that would only work if you weren't developing something using entry
points to tie an extensible application or framework together. And that
didn't use egg metadata for anything else. But if all you used it for was
dependencies, then that might make some sense... except that it would also
mean that nobody else could *use* your stable version to do *their*
development, which is one of the goals here.
It's perhaps unfortunate that egg's bundle package management with
generic meta-data management.
It's not really bundled - integrity checking and inter-package linking do
not a package manager make. EasyInstall is the (rather primitive) package
manager, and it only comes into play if you are manually installing
things. For the normal apt-get scenario, EasyInstall isn't involved, with
the possible exception of applications like Trac installing plugins into
their private plugins directories.
I know that package management uses
meta-data, but if it was implemented as one wrapped around the other,
you could "peel the package management layer off", leaving the metadata
management there, and you could produce clean deb's using only Debian
package management, but with the egg meta-data management still in
place.
There's nothing to peel off, unless you believe that it's wrong to have
runtime integrity checking. That's all the egg runtime does, unless an
application developer codes up something special for their application by
wrapping easy_install. When an application says to the runtime, "I need
foobar", the runtime just checks if foobar is there. This is useful for
optional feature dependencies, for example, and safe integrity checking is
definitely needed for apps with plugins (which is what eggs were originally
invented for: extensible Python applications like Zope, Trac, Chandler, etc.).
Vincenzo's idea about having easy_install be able to wrap apt-get is
interesting, in that it would allow apps that want plugin installation to
satisfy dependencies via system packages that *aren't* already installed,
but in practice I think such applications are going to be restricted to
local or specific-user installation by the very nature of their intended
uses, so being able to have easy_install get automatically hooked to
apt-get doesn't seem necessary to me. In the Debian world, easy_install
should probably only be used for user-private and/or system-local
non-Debian stuff, including apps' private plugins. (And if you did want
easy_install to be able to use apt-get to satisfy dependencies, it probably
means you should just be using something like easy_deb instead, to build
debs of the stuff you want.)
Reply to: