On Mon, 2006-01-02 at 18:31 +0100, "Martin v. Löwis" wrote:
> Joe Wreschnig wrote:
> > 1. Stop compiling .pyo files, entirely (I'm hoping for little
> > argument on this).
>
> I agree.
>
> > How?: compileall.py:57, - cfile = fullname +
> > (__debug__ and 'c' or 'o') + cfile = fullname + 'c'
>
> This is the wrong solution, though. If Python isn't invoked
> with -O, the extension will always be 'c'. The right solution
> is to invoke compileall only once, without -O.
>
> If you apply the proposed change, the .pyc files will be generated
> twice (once for without -O, and once for with -O), you will end
> up with the .pyo file named .pyc.
D'oh, thanks. Then the correct place to patch this is in dh_python's
postinst generator; it's still trivial, though.
> > Disadvantages: * They waste disk space; they use about as much as the
> > code itself.
>
> That's not true (or, rather, I disagree): they don't "waste" the
> disk space, and the code is not "about" the same as the source code
> size. Rather, the size varies: for Python 2.4, htmlentitydefs.py
> is 18054 bytes, and htmlentitydefs.pyc is 6405 bytes (which is
> roughly a third).
>
> > * It's still far too easy for modules to be regenerated for the wrong
> > version of Python; just run the program as root.
>
> I don't understand. What precisely would root do, what precisely
> would happen, and why would that be a problem?
>
> Currently, modules are installed with one for each Python
> version. So if root runs a program, the Python interpreter picks
> up the modules of that Python version, for which .pyc files are
> already generated, so nothing is regenerated when root runs
> a program.
This is not the case; many Python modules are
in /usr/share/program, /usr/lib/program, or an otherwise unversioned
directory.
> Even if you have a package which is installed just in a shared
> way (e.g. in site-python, not site-packages), root would
> generate .pyc files for some Python version, yes. Why would
> that be a problem?
If generating the wrong pyc files is not a problem, why do we care about
generating them at all? Either
1) .pyc files are important. Then we should be sure we're compiling
then right. This doesn't happen under the current system.
2) .pyc files aren't important. Then it doesn't matter if we
compile them at all. But not compiling them is not allowed with
the current system.
The current system goes against both sane conclusions; I advocate 2
(because I think it's easier, and I don't think pyc files matter), and
Josselin Mouette has made some excellent progress towards 1.
> Even if generating bytecode files during installation is
> suppressed, why would that make it harder for root to
> regenerate the modules for the wrong version?
> Why
(Did you lose a sentence here?)
As I said to Josselin, one goal with the proposal was minimal changes to
Python's code initially, so discussion could be around the usefulness of
bytecode rather than the details of a particular patch. I have no
problem patching Python to not generate modules at all unless requested.
>
> > * .pyc files are not really architecture independent.
>
> Right. Fortunately, they are created installation, instead of being
> shipped in the debian package. As you say, they are
> platform-dependent only in border-cases.
But they're also placed in /usr/share, which can be shared across many
systems, so this doesn't help.
> > Another note: Currently, Python policy is based around the assumption
> > that .pyc files are valid within a single minor Python revision. I
> > don't find any evidence to support this in the Python documentation.
>
> Nevertheless, this is a promised guarantee. If you wish it documented,
> just file a bug report.
Okay.
http://sourceforge.net/tracker/index.php?func=detail&aid=1395511&group_id=5470&atid=105470
> > How?: dh_python should not call compileall.py unless give some
> > special flag. Python policy 2.5 should change "should be generated"
> > to "may be generated."
>
> What's the point in having a policy, then? The policy should either
> mandate it (and specify additional conditions if it's not an
> all-or-nothing deal), or forbid it.
Why? If the speed matters in some case, by all means use it! But don't
do it by default, because of the disadvantages of pyc files. This is
similar to how Debian policy suggests -O2 by default, but allows
programs that do benefit from more aggressive optimizations to use them,
despite the disadvantages in doing so.
> > 3. Python dependencies should be loosened (and here I expect a
> > flamewar).
> [...]
> > How?: Strike the third paragraph from 3.1.1. This would also negate
> > the fifth paragraph, which outlines a hypothetical overcomplicated
> > solution to the same problem.
>
> So you are talking only about *programs* here, not libraries?
> And then only about programs shipping private modules?
If I could get agreement that bytecode was irrelevant for programs, that
would still be dozens fewer packages to migrate during the next Python
transition, and that would be great. But the proposal was intended for
both.
> That would be fine with me, but
> - your proposed change does not work (on its own). You would also
> need to mandate that such modules must not be installed
> into pythonX.Y/site-packages (else the application breaks
> after a Python upgrade),
This amounts to saying "the program must work." While that's true, I
don't think it needs to be in policy.
> and you should also mandate that
> such packages don't create bytecode files during installation
> (else those bytecode files become redundant after an upgrade).
Right. Modules that intend to be shared between Python versions can't
generate bytecode (unless we have something like python-support).
--
Joe Wreschnig <piman@debian.org>
Attachment:
signature.asc
Description: This is a digitally signed message part