Re: handling /usr/local/lib/python2.x/site-packages in sys.path
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mar 11, 2008, at 6:36 PM, Floris Bruynooghe wrote:
On Tue, Mar 11, 2008 at 09:45:21AM -0400, Barry Warsaw wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mar 11, 2008, at 3:43 AM, Matthias Klose wrote:
Currently Debian's python has /usr/local/lib/python2.x/site-packages
in sys.path allowing for installation of local modules. Barry did
point out that this conflicts with a python installation, where
/usr/local is the default prefix, and the system python gets modules
from the local installation. Some suggestions were made to fix
this:
- add an env var to not include /usr/local/lib/python2.x/site-
packages
when the env var is set. Keeps standard behaviour without
modifications and allows people to remove it from sys.path. But
requires the user to know about that option.
- add another path (e.g. /usr/local/python/lib2.x/site-packages),
and remove the /usr/local/lib/python2.x/site-packages path after
the next release. Does provide an upgrade path, but doesn't solve
the probem immediately.
Thanks for raising this issue Matthias. There are a couple of other
points I'd like to make.
This isn't /just/ a problem for Python experts or developers.
Sometimes
we recommend that Ordinary Users install Python from source, say to
use
an application that isn't supported by a distro's packaging system.
There can be lots of reasons for this: maybe the app is in beta
testing,
maybe they want to try things out without affecting the system
version,
etc.
Python developers may not be Debian experts and may not know about
this
system peculiarity. So when we tell people "just do configure; make;
make install" we can actually do inadvertent damage to their Debian
system.
Problem is that both are very natural, having local packages being
picked up by the system python in
/usr/local/lib/python2.X/site-packages as well as being able to
install a new python with a prefix of /usr/local.
I asked quite a few Python people about this at Pycon 2008, and I
raised it in a Python packaging session with Matthias and a few Fedora
people.
I actually think this is natural only for Debian users, and not the
wider Python or Linux crowd. The Fedora people don't do this, nor
does Gentoo AFAICT. It surprised quite a few Python folks I spoke
with, even some that use Debian or Ubuntu.
Given this choice I'd prefer the second option as well, it seems more
natural then the first and an upgrade path is important (Wichert's
argument about the first having problems with su is also important).
It would be good if the distutils could be changed to install local
packages there by default then, so as to provide the least surprises:
just "python setup.py install" as root would do the correct thing.
I agree that for Debian and Ubuntu, where this tradition is firmly
ingrained, the second option makes the most sense. It lets you have
the functionality you want without breaking the expectations of the
wider Python community. Everyone can live together peacefully! :)
I'm not sure your other suggestion make sense from a default distutils
perspective. The reason is that distutils doesn't see the world that
way, and it really doesn't know anything about the rest of the file
system, other than the paths it gets from Python or that you tell it
about. So having it install in Python's own site-packages by default
still makes sense, but it should be easily configurable to do
otherwise, either through command line switch or other mechanism.
That's the case today though :).
When looking at the grand scale of things the change for the system
Python to look for local packages in
/usr/local/python/lib2.X/site-packages (or
/usr/local/python/site-packages2.X?[1]) is probably best done upstream
so it is on all systems the same. The build system could do this when
a prefix of /usr is passed to configure.
I don't think it makes sense to get this into upstream because this is
really a distro question. What about MacPorts installing into /opt/
local? What about if I install from source into ~barry/development/
python2.6? None of those should have anything to do with /usr/local.
What confuses me is that this doesn't appear to be a convenience
based
on permissions,
It has nothing to do with permissions. The sysadmin should never
install things into /usr/ directly, /usr/local/ is their playground.
This is Debian policy (which is fine), but I don't think all distros
agree. I'm not a distro guy though. :) Mattias, didn't the Fedora
guys say they were going to try to create a mailing list for
discussing these kinds of cross-distro issues?
I'm also uncomfortable with the way that Debian accomplishes this: it
hacks Python's standard site.py. There really should be a better
way to
do this (I have to think about this a bit though).
In the python we ship to our customers (living in /opt/vendor/python2X
as per FHS) we place a .pth file in the site-packages directory to
accomplish extra locations on sys.path (pointing somewhere else in our
/opt/vendor hierarchy). Maybe
/usr/lib/python2.X/site-packages/debian.pth would be a better way to
handle this on Debian? Or are there reasons not to do this? Although
the suggestion I make above, changing it upstream, would make this
unnecessary.
I just looked in Ubuntu Gutsy and Hardy, and in those distros, there's
is actually a hack to site.py to add /usr/local explicitly. Maybe
Debian does it differently, but I don't have a system to check. I do
think using debian.pth would be a much better way to do it than
hacking Lib/site.py from upstream.
[As an aside, I find what happens with python-support much worse. I
regularly waste time re-discovering that a whole load of system
installed packages can't be found in /usr/lib/python2.X/site-packages/
but live somewhere in /var/lib/python-support/. It's a violation of
the FHS as well, /var/lib/ is for program state information, .pyc's
are not state).]
Right, pycs are cache! :) I can't comment on the rest of this
paragraph. ;)
Cheers,
- -Barry
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
iQCVAwUBR+mHb3EjvBPtnXfVAQL2DwP/XGIQFmXcwWabeo4KtLBKvbk3AQOENqcd
ndS4dK1v6oijNC98mc14syjR16wszU9+LPwYTgUXIkP7Q9JAWSIE3/A+UL0lKxet
4IDEswdjcmRMDI27UTSuf09fin+YJ2icuIwTpKbuXbH3tFxxvP/wcwFZSFmMPx0L
/2wpSSW15gk=
=4cYg
-----END PGP SIGNATURE-----
Reply to: