> On Jan 22, 2016, at 11:51 AM, Scott Kitterman <debian@kitterman.com> wrote:
>
> On Friday, January 22, 2016 10:54:54 AM Donald Stufft wrote:
>>> On Jan 22, 2016, at 10:36 AM, Piotr Ożarowski <piotr@debian.org> wrote:
>>>
>>> to be honest, I still don't know what you're asking for. What do you
>>> want us to do? Patch 2.7's distutils?
>>
>> Essentially, ensure that setuptools not distutils is used in a setup.py.
>> There are generally three kinds of setup.py files:
>>
>> 1) Ones that use setuptools unconditionally - These ones you just leave
>> alone, they are already correct and you should already have a build depends
>> on python-setuptools. 2) Ones that conditionally use setuptools - These
>> ones you just need to satisfy whatever condition the setup.py uses to
>> enable setuptools. Typically this is just checking if setuptools is
>> importable but sometimes they use environment variables or similar. 3) Ones
>> that use distutils unconditionally - These ones you switch to making them
>> use setuptools instead of distutils.
>>
>> Now, that’s the high level overview, there’s an easier, more automatic way
>> that could maybe just be added to pybuild (Not sure exactly how pybuild
>> works) where instead of invoking the setup.py as:
>>
>> python setup.py install (or whatever commands/args you’re passing)
>>
>> You do it as (taken from pip):
>>
>> python -c "import setuptools,
>> tokenize;__file__='$PWD/setup.py';exec(compile(getattr(tokenize, 'open',
>> open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))” install
>> (or whatever commands/args you’re passing).
>>
>> The thing is kind of ugly, but that will install things using setuptools
>> (just like pip does) regardless of if it imports setuptools or distutils in
>> it’s setup.py file.
>
> I tried this and it works, but what's the big deal?
>
> It provides a PKG-INFO file that has identical content to the old egg-info file,
> an empty dependency links file, and a top_level.txt file with the one line in
> it.
>
> Why is this better (I'm not a huge upstream developer of Python stuff, but I do
> do some and I don't see what this gets me)?
>
> Scott K
This a fair question:
1) distutils is effectively frozen and all improvements to the build system
are either going to be inside of setuptools, or will be opt in build systems
that (probably) won't rely on setup.py at all. This includes newer metadata
formats and the like as well.
2) That top_level.txt is an important file, without it we have no idea that
$SITE-PACKAGES/requests is in any way associated with the metadata file
requests-2.9.1.egg-info, that top_level.txt tells us what the top level
import names are for a particular project.
3) It slipped my mind that you have to pass an additional flag to setuptools
right now to get the full file list (pip passes that flag unconditionally)
however I'm going to poke setuptools to see about getting them to add the
record file unconditionally to the .egg-info directory so it doesn't
require the --record flag. (Although Debian could add it earlier if they
wanted, but it's fine to wait for setuptools to change here).
4) Switching it to a directory allows us to add additional files/information to
the Python level metadata, like the sort of divergent thread about getting
pip to stop mucking with OS owned files, the most likely path forward is to
drop an INSTALLER file inside the .egg-info directory that has some special
value ("system"? "os"? Not sure) which pip will take as a signal that it
really should not touch.
5) In pip, because of #2 and #3 a regular distutils installed package (which
gives you an .egg-info file, not directory) we have now way when we're told
to install a package named "foo" to associate that with any files on disk.
This means that we're currently lying when we tell a user we've uninstalled
a distutils installed package, we just remove the .egg-info file but leave
all the other files behind. We've deprecated the ability to do this, and we
tried to remove it all together in the recent pip 8.0 release, but we had to
roll it back because it caused too much breakage, largely due to OS packages
that were using distutils installed metadata.
On the surface, it may seem counterproductive for Debian to help pip get
better metadata about what files belong to a package so that we can
uninstall it, but we're going to solve the "people modify system files with
pip when they shouldn't do that" more directly and there exist use cases
where it's perfectly fine to break your system in this way, because it's an
emphereal system (think CI servers) where you just want to get the newest
version of a package (maybe just one package) into the system in order to
run some tests before you throw the whole server away or something equally
as reasonable.
-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Attachment:
signature.asc
Description: Message signed with OpenPGP using GPGMail