Re: Amend Debian Python Proposal to Include More Python Metadata?

On Jan 21, 2016, at 9:32 AM, Paul Tagliamonte <paultag@debian.org> wrote:

Hey Donald!

As far as using pip to do stuff system-wide, I wrote thoughts on http://notes.pault.ag/debian-python

I just want to be clear, that I totally agree with this. The reason I want to

ensure we can enable this isn't because I think doing so is a great idea and

everyone should be doing it. People *are* doing it and we (pip) have to be

careful about what we make impossible to do because a possible result of going

too break happy is people just never upgrade and things start to stagnate

again.

So for pip at the system level, my goal is try and push people away from it as

much as we can without flat out making it impossible *and* try to make it as

"safe" to do so as possible when people decide to do it anyways.

As for the rest of it, distutils is actually concretely shitty, and replacing it with setuptools
sounds sane. Sounds like a solid idea.

Perhaps we can add a Lintian warning for using distutils; it's really nasty cruft,
and it's going to bite the archive sooner rather than later.

As far as options for how to implement this goes, I'm not sure what best fits

with Debian so I'll leave that up to y'all, but here are a few options that I

can think of.

A lintian check is probably a pretty good thing, I would guess that it would

specifically check for the existence of a .egg-info or .dist-info directory

(or check to the non-existence of a .egg-info file).

In pip the way we make this work is with a bit of a gross little thunk that

ensures setuptools is imported prior to setup.py being invoked which forces all

setup.py calls to use setuptools instead of distutils even if they import from

distutils.

Of course, many setup.py already unconditionally use setuptools, and many more

conditionally use it (typically falling back to distutils) so for many projects

it may be enough to just add python-setuptools to build-depends. For projects

that unconditionally use distutils, either the pip way or patching setup.py

might be a reasonable way forward.

A more controversial way that comes with possibly some extra benefits (which

Debian may not care about) is to use ``pip`` itself as the build tool rather

than directly invoking setup.py. In this pip would be responsible for mucking

around with the distutils/setuptools mess instead of that needing to be handled

by Debian specific tooling. You could even leverage this to remove some of the

runtime dependencies on python-pkg-resources and speed up some python using

software if you let pip build a wheel as part of the buld process [1][2][3]. I

suspect pip would need to grow some additional options to make it suitable for

you in this role, but I would be more than willing to help if this is desired

(or not, this proposal isn't about trying to ram pip down anyones throat!).

[1] To be clear, this isn't about adding wheels for everything to the archive,

it would only be using Wheel as an intermediate step in the build process.

[2] Console scripts generated by setuptools entry wrappers when installed by

setuptools have a dependency on pkg_resources at runtime, however those

same scripts, when installed by pip from a wheel, do not. This would enable

removing the runtime dependency on pkg_resources for anything that only has

it for console scripts.

[3] Import pkg_resources is not the fasest thing in the world, it has to scan

every directory on sys.path looking for things to activate and it comes

with a bunch of side effects. This happens implicitly for any project using

console scripts.