Re: [Distutils] formencode as .egg in Debian ??
Phillip J. Eby wrote:
> At 06:33 PM 11/22/2005 +0100, M.-A. Lemburg wrote:
>
>>Phillip J. Eby wrote:
>>
>>>Yes, it's true, zipfile import processing is faster than normal import
>>>processing;
>>
>>Only after *all* ZIP files on sys.path have been scanned
>>for their contents. The more you add to sys.path, the longer
>>Python takes to startup.
>
>
> This is simply not true. If you don't believe PEP 302 and site.py, measure
> it for yourself. The *only* addition to startup is the time to actually
> read the .pth file and append the entries to the list.
>
>
>>What's worse is that the slow-down affects the whole Python
>>installation - each and every application using Python will
>>have to scan all these ZIP files in case it tries to import
>>a non-existing module or one which it finds late on sys.path.
>
>
> And how often do programs attempt to import non-existing modules along
> performance critical paths?
Every single time you fire up Python and the user has not
installed a module called "sitecustomize" (which is deliberatly
not shipped with Python), Python will scan the complete sys.path
for this module... and that's just one example.
It is rather common in Python code to test for the availability
of a faster variant by trying an import (e.g. for XML parsers)
and then falling back to some slower emulations.
> Note by the way that "scan all these ZIP files" is a misleading term in any
> case - the files are not "scanned". They are opened, and a small amount of
> data is read from the end of the file. Nothing that I would consider
> "scanning" is involved.
The data read from the end of the file is the directory
which is decoded using marshal functions. You normally
call this scanning data.
Like Martin said: you always have to read the whole ZIP
directory - even if you're just interested in a single
module with the file.
Actually loading the module then requires decompressing
the code which takes a whole lot longer than just reading
a file from the file system.
In summary, things get slower when importing from ZIP files;
it really only makes sense for applications that have a long
run time and where startup is not that important, e.g.
Zope et al.
The main argument for using ZIP imports is to easy
distribution of complete pure-Python packages, not
a performance gain. You'd use one of the freeze tools
for that, e.g. mxCGIPython which creates a single
file Python interpreter which has a really good
startup time due to the fact that the Python lib
is embedded into the Python executable as static data
and then loaded on demand by the OS as needed.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Nov 22 2005)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
Reply to: