Re: python packaging infrastructure
Steve Langasek writes:
> On Fri, Jan 20, 2006 at 10:47:19AM +0100, Matthias Klose wrote:
> > Steve Langasek writes:
> > > On Wed, Jan 18, 2006 at 01:06:39PM +0100, Matthias Klose wrote:
>
> > > > the design decision of putting the binary-all python packages in a
> > > > separate directory into /var/lib/python2.x/site-packages has some
> > > > problems when supporting packages with extensions (a proposal beeing
> > > > made on #irc was to keep the extensions in the standard path).
>
> > > If they are in different directories, then foomod.py already doesn't enjoy a
> > > privileged location in the current path; so whether they are symlink-farmed
> > > or not, there must already be handling in place to locate foomod.py, so it
> > > doesn't much matter which particular directory it's located in.
>
> > > Perhaps you are arguing that a package should be able to ship
> > > /usr/lib/python2.3/site-packages/foo/fooext.so and
> > > /usr/lib/python/site-packages/foo/foomod.py, and have the infrastructure
> > > make foomod.py (or foomod.pyc?) available in
> > > /usr/lib/python2.3/site-packages/foo. I'm not sure why this would be
> > > advantageous?
>
> > > > So how does python-support handle packages, where the extension
> > > > module is split out in it's own binary package?
>
> > > If they're split out in separate binary packages, then surely the extension
> > > package doesn't require any special handling, and the module package can be
> > > managed as before?
>
> > yes, another bunch of packages which then cannot be handled by the
> > packaging infrastructure.
>
> Uh, no, I meant "can be managed as before [by python-support]".
>
> > > (since obviously it's not sensible to split
> > > /usr/lib/python2.3/site-packages/foo between two separate packages...)
>
> > obviously?
>
> > - In the past packagers were encouraged to split packages this
> > way to save disk space (at least on the ftp mirrors), if the size of
> > the arch independent part exceeds the arch dependent part significantly.
>
> > - packages are split this way, if the package does depend on other
> > libraries or packages, which are not always needed (i.e. have a
> > dependency on X in a separate package, have a separate package for
> > each database adaptor, built from the same package source).
>
> > - packages are distributed upstream this way, so you have different
> > sources
>
> > sounds rather sensible to split these.
>
> Ah. So this happens in practice? These do all seem to be reasonable;
> apparently my "obvious" answer was wrong. :)
>
> > Coming back to:
> > > Perhaps you are arguing that a package should be able to ship
> > > /usr/lib/python2.3/site-packages/foo/fooext.so and
> > > /usr/lib/python/site-packages/foo/foomod.py, and have the infrastructure
> > > make foomod.py (or foomod.pyc?) available in
> > > /usr/lib/python2.3/site-packages/foo. I'm not sure why this would be
> > > advantageous?
>
> > I.e. in version 1, a package is implemented as a pure python module,
> > the package maintainer uses the new packaging infrastructure, version
> > 2 implements some functionality in an extension module to speedup
> > things. Why does a package maintainer have to change the packaging
> > infrastructure?
>
> Er... because the nature of the package has changed? There are going to be
> changes required here anyway; adding a .so to the package means it can no
> longer be arch: all, can no longer be invariant wrt multiple python
> versions, has to build the module once for each flavor of python supported,
> etc. I believe we should provide the best possible packaging tools to
> simplify this for maintainers, but I think that expecting a maintainer not
> to have to do any work to convert from a pure-python package to a package
> with an extension is unrealistic.
no, but it breaks unrelated (maybe binary all) packages as outlined in
the second scenario below.
> Btw, perl has had split arch:any/arch:all paths for modules for years, and I
> haven't heard anyone complaining about it; I'm really not sure why python
> should be different.
- Perl seems to have a more stable ABI for it's extensions
- Perl doesn't have version dependent byte code files, which are not
compatible across versions.
> > Make it worse, if a pure python module offering some kind of plugin
> > structure uses the infrastructure, another package (maybe even
> > maintained by somebody else) chooses to implement the plugin as an
> > extension enforces the other maintainer to drop using the packaging
> > infrastructure.
>
> This is because the plugin search path will be relative to the directory the
> python module is installed in? Can you give me some example python code
> that would fail in this scenario?
why another one? there's one which shows this behaviour. it is enough
to show that it doesn't work.
Matthias
Reply to: