Re: Packaging pypy
Hello Thomas,
It could turn every "module.py" into a "module.so" with more or less
dubious benefits.
Note that there are a few differences between compiled and pure Python
modules. E.g.
- Tracebacks to errors won't show code from compiled files.
That is only if the ".py" file doesn't exist, right? Why wouldn't it. To
me, compilation is not about removing the source code. Not at all, it's
only about acceleration.
- Functions assigned to a class attribute get bound to instances (i.e.
get automatically passed self) if they're defined in Python, but
compiled functions do not. I had to fix something recently where a
standard library function had moved from pure-Python to C, and broken
code that bound it to a class.
Nuitka's "compiled_function" is way more compatible that "PyCFunction"
if that is your concern. It has "func_code", etc. and works file in
descriptors. Monkey patching, use in properties, metaclasses, whatever,
is supposed to work with it just fine, and tested in the CPython test
suites I run. In principle the only differences are:
a) The "func.func_code" has no bytecode object (not going to change)
b) The "func.func_defaults" are not there (changing later this year)
It's really supposed to be as good as before. Now already. I do not pass
the "hg" test suite with "compiled hg" yet. But once it does (needs at
least func_defaults to work, and I believe I do not enforce enough
evaluation order apparently yet, these things are not well covered by
the test suite), that kinds of proves a lot, because hg is really very
dynamic code.
BTW: I am not sure a compiled "hg" itself is beneficial, because it's
likely performance bound by IO.
- I can arbitrarily add extra attributes to instances of Python classes,
but not to instances of compiled classes.
Nuitka is also more compatible here, although I am not so sure, if
compiled classes of any kind have that problem.
Also, I occasionally go and tweak installed Python files, which is
obviously impractical if they have to be translated and compiled.
True enough. I would hate it too. Lets say, the meta path importer that
would be hooked in site.py checks dates. Then you would only become
slower (use the modified version), and/or would have it compiled again
at import time (slower to start, once, but permissions would be an
issue, you like edit as root, but you run as mortal user).
I don't think it would have to be impractical. I wouldn't want to live
in a world, where Python programs become hard to modify.
I know you're not suggesting we turn this on tomorrow. But I think
replacing pure-python modules with compiled modules should always be a
deliberate choice, not something distributions do automatically.
Fair enough. Even if one can approx. the amount of transparency of the
measure, it e.g. should be considered if it's useful at all, by the
maintainer of a module or program, and would be something to turn on via
configuration of course.
StringIO already has compiled equivalents, of course - cStringIO, and
the compiled io package in Python 2.7 and 3.x.
I just meant to give an example. I believe PyPy does port C implemented
modules to Python, because they can then execute it faster. To a
compiler, be it JIT or not, Python is potentially better input.
BUT: Right now, my priority is to get the release out that has a Debian
package, and to find a sponsor for it, so Nuitka is available to Debian
users to experiment with it.
Still I am already in a position to state that Nuitka created ".so" and
".pyc" files have no practical differences except speed of creation and
sizes. So on a design level, these could be treated the same.
This "being compatible" part is currently basically the only priority at
all. I have this plan, where that level of compatibility is the
overarching first goal to achieve, even before performance numbers are
even near impressive or measured a lot.
All, I would like now and here, is just to understand, how PyPy is fit
into things, and if that's a possible path for deeper Nuitka integration
later on too, that would be great.
If it's not, no problem, at all. I am no big fan of over complicating
one solution, because potential other uses. I wouldn't ask for it. I
just saw that PyPy suggested "Alternate" and not "PyPy" and a hint that
they of course would like to have a universal solution.
Yours,
Kay
Reply to: