TL;DR: Let's re-enable the ensurepip module in Python 3.4, and possibly
address some usability issues. We should descend en masse on Montreal and
stage a revolt at Pycon. :)
Python 3.4 has an `ensurepip` module[1] which implements the specification in
PEP 453 regarding the explicit bootstrapping of pip in Python installations[2].
This is promoted as a boon to users, especially on platforms without OS
provided package managers, i.e. not Debian.
The PEP makes some recommendations for downstreams[3], which we do not
currently adopt, and maybe shouldn't fully. Our current Python 3.4 package
disables ensurepip at build time.
The previous discussion on this subject[4] was extensive, and it may not be
worth rehashing that. I just reviewed the thread via Gmane, and AFAICT there
is some general consensus:
* pip should not be used to install packages "globally", i.e. in the system
Python's dist-packages.
* pip is an essential tool for virtualenv users (i.e. developers).
* Debian will never package everything that's on PyPI, and even those that it
does package can be out of date, for both valid (e.g. it breaks something)
and lazy[5] (maintainers haven't gotten around to it yet) reasons.
The current situation in the Python 3.4 package is suboptimal because:
% pyvenv-3.4 /tmp/zz
Error: Command '['/tmp/zz/bin/python3.4', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1
Although `virtualenv -p python3.4 /tmp/zz` does work.
At the very least I would like to re-enable ensurepip so that pyvenv-3.4 works
properly. I am okay with the pip that gets installed in a pyvenv is the one
from the embedded wheel instead of the system pip. I don't think this
necessarily violates Debian policy $4.13 against "convenience copies" since I
view virtualenvs as isolated environments which *might* share some Debianisms
(e.g. if --system-site-packages is used) but *should* be considered entirely
separate development environments.
Questions and issues:
Q: What should `python3.4 -m ensurepip` do?
IMHO, outside of a virtualenv, it should either fail, or it should do like
command-not-found would do if you ran this at the shell without python-pip
installed:
% pip
The program 'pip' is currently not installed. To run 'pip' please ask your administrator to install the package 'python-pip'
pip: command not found
It already *will* fail unless you run that as root, but I think even if you do
run it as root, it should fail outright, even if you did
`python3.4 -m ensurepip -U` in a vain attempt to upgrade the system's pip.
In some ways this is aligned with PEP 453's recommendations and in others it
runs counter.
In a virtualenv, `python -m ensurepip -U` should do the right thing.
Q: What about `pip install ... --user`?
That should work! It's one good use case for keeping the python-pip and
python3-pip packages.
Q: How can we tell if we're in a virtual environment?
Too bad there's no `import venv; venv.am_i_in_a_freaking_venv()` API. :(
a) Look at the envar $VIRTUAL_ENV. bin/activate sets this, but it may not be
a perfect determination. E.g. if you run /my/venv/bin/python without
activating it, that environment variable won't exist.
b) Look at sys.real_prefix. This exists in virtualenv created environments,
but not in pyvenv environments.
c) Check if sys.prefix == sys.base_exec_prefix. This should be False in
pyvenv environments, although it is True in virtualenv environments and in
the system Python.
-----snip snip-----invenv.py
import os
import sys
if 'VIRTUAL_ENV' in os.environ:
# The user is living in an activated virtual environment.
sys.exit(0)
if hasattr(sys, 'real_prefix'):
# Unactivated use of $python in a virtualenv created environment.
sys.exit(0)
if sys.prefix != sys.base_exec_prefix:
# Unactivated use of $python in a venv environment.
sys.exit(0)
sys.exit(1)
-----snip snip-----
% python3.4 /tmp/invenv.py && echo "yes" || echo "no"
no
% /tmp/venv/bin/python /tmp/invenv.py && echo "yes" || echo "no"
yes
% /tmp/virtualenv/bin/python /tmp/invenv.py && echo "yes" || echo "no"
yes
% source /tmp/virtualenv/bin/activate
(virtualenv)% python /tmp/invenv.py && echo "yes" || echo "no"
yes
(virtualenv)% deactivate
% source /tmp/venv/bin/activate
(venv) % python /tmp/invenv.py && echo "yes" || echo "no"
yes
Q: Should we make python{,3}-pip a Depends of python3.4?
PEP 453 recommends this for downstreams, but this will create a dependency
loop. Fedora recognizes this too. Our Gmane thread briefly touches on
whether a Recommends or Suggests is better or worth it. I don't know.
I: Should we follow Fedora?
Fedora is discussing some of these issues too[6]. Looks like one of their
devs created an rpm->wheel conversion script so that if you pip install a
package from the archive, it'll get the rpm, convert it to a while and install
it in the virtual environment (IIUC). I don't much like that, so I don't
suggest we take that approach.
Q: What about the other PEP 435 "Recommendations for Downstream Developers"?
* It might be interesting to at some point utilize wheels, but I think that's
far enough out that we should ignore it until we can't. ;)
* Never allow `pip install --upgrade pip` to upgrade system pip.
That's all I can think of.
-Barry
[1] http://docs.python.org/3/library/ensurepip.html
[2] http://www.python.org/dev/peps/pep-0453
[3] http://legacy.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors
[4] http://article.gmane.org/gmane.linux.debian.devel.python/9143 (among others)
[5] Not meant to denigrate any maintainers, so probably better to say
s/lazy/busy/ :).
[6] http://news.gmane.org/gmane.linux.redhat.fedora.python
Attachment:
signature.asc
Description: PGP signature