[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: dh_python and python policy analysis



Le sam 12 août 2006 17:34, Matthias Klose a écrit :
> Manoj Srivastava writes:
> >  policy document. The current version, and future updates, are to
> > be found at http://www.golden-gryphon.com/software/manoj-policy/
>
> unreachable, comments for the posted text follow

doh, that works for me ?!



> >    Extension Module
> >
> >              Extensions are C code compiled and linked against a
> > specific version of the libpython library, and so can only be used
> > by one version of Python.
>
> There should be no reason to link the extension against the python
> library.  Usually many extensions which are developed upstream on
> Windows do link by default to libpython.  Other extensions linking
> against libpython are those with build infrastructure maybe predating
> distutils.  python-semanage is an example (and should not link using
> -z defs).

> Another thing to mention here is a "Python package", a directory
> containing an __init__.py file plus modules and extensions.

yes, that's indeed a thing to add.

> >    Public
> > […]
>
> maybe shorten that to "all directories in sys.path"; not sure if an
> explicit list of directories is needed.

I've no real opinion on that.

> >              Packages may install public Python modules in
> > directories specific to Python packaging utilities -- which specify
> > the directories under which such modules should be droppped, and
> > the the structure of these directories is defined by the utilities
> > themselves. Please note that these directories are not in the path
> > for Python, and are not available for modules to be imported from.
> > At the time of writing, such uility specific directories include:
>
>                                            ^^
>
> >            /usr/share/pycentral
> >            /usr/share/python-support
>
> These location are tool specific and should not be referenced
> explicitely in the packaging scripts (debian/rules)

agreed


> No, not the whole design goal.  Although the document is titled
> "developer's view", the other goals should be mentioned as well.
> These are meant to work around processes in debian which are
> currently suboptimal, but unlikely to change.
>
> […]
>
>    The alternative of dropping the python-foo package and just
> keeping the pythonX.Y-foo packages was not followed anymore (ftp
> master intervention and rewriting of control files).
>
> These are design decisions made for the distribution.  There are
> concerns about some upstream developers that the design of dropping
> the pythonX.Y namespace for packages may not be rebust enough.
>
> Another consequence of the current design: the default python version
> *has* to be installed, other supported versions can be installed
> additionally, not as a replacement.

agreed, this also has to be mentionned.


> >      This can be a single version, or one or more of a list of
> >    non-overlapping ranges. The lowest range may optionally omit a
> > low end, and the highest range may optionally omit an upper end. In
> > other words, the overall range may be open ended. The ranges are
> > often matched to the set of all known Python version that have
> > existed, and the supported set is the intersection of the known
> > versions of python and the range specification.
>
> XS-Python-Version can have the values "current" and "current_ext" as
> well (plus the list of ranges), which will expand to "current", if
> the package does not have any extensions and can be used with another
> python default version without a new upload. It's replaced by the
> version number of the current default version in the Python:Versions
> substitution variable. "current_ext" normally only needs to be used
> for packages having a private extension module. dh_pysupport doesn't
> use this information, but requires the developer to explicitely pass
> the directory containing the extension module.

IMHO current and current_ext are choices of implementations of 
pycentral, which your following explanations truly underline.

[snip]

> >     3.1.2. Depends:
> >
> >      The rules for calculating the dependencies a package has are
> > simple.
> >
> >     1.   If a script invokes /usr/bin/pythonX.Y, than the package
> > must depend on pythonX.Y. This is because no amount of automatic
> > byte compiling would ever get rid of the requirement that
> > /usr/bin/pythonX.Y has to be present for the script to function.
>
> I think, that is too strict.  The current behaviour is depending on
> the dh_python implementation scanning all files for that interpreter
> line.  Consider a package with scripts in /usr/bin: foo, foo2.3,
> foo2.4, calling python, python2.3 and python2.4, which would lead to
> a dependency on all supported python versions.  The scripts work for
> the default python version, for the non-default python versions only
> if the corresponding pythonX.Y package is installed.

then the packager can choose to make those script use /usr/bin/python. 
I've seen very few of those packages, and if those truly exist, I'm 
confident we can solve the problem easily.


> >     3.1.3. Provides
> >
> >      Packages with public modules and extensions should be named,
> > or should provide, python-foo. Pure Python public modules that
> > support all Python versions need not have a Provides field.
>
> ... unless there is an application using a non-default python version
> using this module. or else you require the application depending on
> any indirect dependency of python-foo.

well, I think the latter is still the way to go, else the archive will 
be quite awkward with mixed python-foo and pythonX.Y-foo...


> >     3.1.4. Build-Depends:
> >
> >      If the package provides public extension modules, then build
> > depending on "python-all-dev (>= 2.3.5-11)" shall ensure that all
> > the >pythonX.Y-dev packages are available during building.
>
> Limiting this to public extension modules is not robust, if a pure
> python module hardcodes version information.

true.

> >   3.2. Deprecating "current" in versions supported
>
> There is currently no agreement about that.
>
> >      Currently, the string "current" in the field XS-Python-Version
> > is used by python-central to indicate that the package contains
> > private modules,
>
> that is wrong.
>
> >    and explicitly state that the package is only built for the
> > current Python version, and not for any other version supported in
> > Debian. This is flawed, for the following reasons:
> >
> >      *   The value corresponds to the version of Python the package
> > is currently built for; but in all other cases the semantics of the
> > XS-Python-Version field is to indicate which versions of Python are
> > supported by the package, and indicates compatibility, not the
> > version it is currently built against. So this special case breaks
> > the semantics of the field.
>
> that is the reason that "current, >= 2.3" is supported.

then in that case (see my remark above) current beeing only used by 
pycentral, there is no need to hardcode current as an acceptable value 
in the policy. I've provided you tools that looks to the generated 
depends crossed with the packages build-deps that are already able to 
say which packages needs to be binNMUed/reuploaded, ... and that does 
not need its value to work.

> >      *   By hijacking the XS-Python-Version field to indicate the
> > version of Python it is currently compiled against, it hinder s the
> > propagation of compatibility information, so the ability of the
> > maintainer to indicate the range of Python versions this package is
> > compatible with.
>
> Not sure I understand that paragraph, what do you want to say?

when we elaborated that part, we thought that "current" was exclusive 
with other informations, wich is also what the few doc available let to 
think. your previous remark just cancel that §.

> >      *   The information conveyed by this field is redundant; it
> > should be clear that the package contains private modules, based on
> > the directories the modules are shipped in, and also the fact that
> > it ought to build depend on python-dev and not on python-all-dev.
>
> python-all-dev does not differentiate between "current" and
> "current_ext";  dh_pysupport has an implementation decision to hide
> that information as a parameter to dh_pysupport.

already answered to that bit in the other mail.

> That information should be visible in the control file.

why ? if a private module is there, then the package needs to depend 
upon some kind of python range (and *not* pythonX.Y, that's for script, 
or packages *specifically* built for a pytho version, hence not in the 
scope of the policy).

if a package Depends: python (>= X.Y), python (<< X'.Y') I can tell you 
for sure there is a module (or an extension) in there. beeing private 
or public is quite irrelevant for the packaging and upgrade matters. It 
only matters to the tool that does the byte compiling (mostly pycentral 
and pysupport, that know where to look already).

> >      *   The semantics of "current" are not fixed, since they
> > depend on the contents of the package python-defaults, and are ill
> > suited for the debian/control file.
>
> again, there's nothing wrong about that, as you can use current
> together with a list of support versions.

same remark as above, that's our mistake, that § has to be removed too.


> >     3.3.1. Supported versions
> >
> >      If a script invokes /usr/bin/pythonX.Y, then the version
> > supported by the source package (XS-Python-Version or
> > debian/pyversions) should be restricted to X.Y, assuming that the
> > field is being provided. Or else, it should be set to the list of
> > python versions that the script can support, or "all". [36][6] If
> > there are separate scripts that invoke different versions of
> > Python, then all these versions must be in the Depends field -- if
> > you still want to continue packaging instead of just shooting the
> > upstream.
> >
> >      No script must use /usr/bin/python if it needs a Python
> > version strictly greater or strictly lower than the current
> > version.
>
> see above (packages having foo, foo2.3, and foo2.4 scripts), that
> requirement should not be hardcoded in that case.

same answer ;) I do think it's up to the debian maintainer to merge 
those scripts if possible. Moreover a binNMU on the package (if 2.3 is 
not supported, and that the package uses pyversions -s) will fix that 
trivially anyway, wich remains quite acceptable.

> >   3.5. Private Extension
> >
> >      These are compiled files linked to python libraries, and kept
> > in a
>
> no, in most cases, any extension needs to be linked against
> libpython.

sorry I miss that mistake by rereading manoj's doc, you're obviously 
right.

> >    by the binary package (XB-Python-Version field or the file
> > .versions) is set to that version (copied from XS-Python-Version).
> >
> >    If the current version is not supported, this field it set to
> > the minimum version actually supported by the module.
>
> what do you mean?

if the version range is e.g. 2.4- with current beeing 2.3, then ... then 
it's indeed wrong. at least I don't understand the rationale anymore 
here. needs a fix.

> >      Depending on the packaging utility used, the modules live in
> > either /usr/share/python-central or in
> > /usr/share/python-support/$package.
>
> that is wrong. 1) there's nothing wrong having these still in
> /usr/lib/pythonX.Y/site-packages.  2) please avoid naming of these
> directories in the document. these should be considered private
> directories for the tools. For the case of pycentral, you can get the
> directory name using "pycentral pycentraldir <package name>".

aggreed.

> >      Official pure Python modules generally live in a different set
> > of
>
> official?

like in upstream's

is there any python modules that go in /usr/lib/python2.3/ directly ?



> >     3.6.1. Byte compiling
> >
> >      In the common case of pure Python modules in unversioned
> > public module directories, tools exist to help byte compile the
> > pure Python modules for all versions of Python installed on the
> > target system. In case of pure Python modules in versioned public
> > module directories, byte compilation is up to the package scripts.
>
> maybe that's not the best place to mention /etc/python/debian_config,
> but scripts byte-compiling files should honor the byte-compile
> property. packages should only byte-compile the files belonging to
> the package, or else error message for byte-compilations are reported
> for random packages.
>
> 3.8 missing: Packages using embedded python interpreters
> (libapache2-pythonX.Y, which should not be collapsed as
> libapache2-python), vim, and maybe other packages.

agreed


> >     4.1.1. rtupdate script invocation
> >
> >     1.   in the pre-installation phase of the python package, the
> > package supplied scripts are called with the parameter:
> > pre-rtupdate <old runtime> <new runtime>
> >
> >          A failure in any script results in the failure of the
> >        pre-installation script of the python package.
> >
> >        [42]Note   Whether or not all scripts are run, or the
> > process aborts at the first failure, is still under flux
> >
> >        Since such a failure of a script would leave all packages
> > whose pre-rtupdate has been run in a dangling state, a bug in a
> > pre-rtupdate will always be a critical bug. Be very very careful
> > when working on a pre-rtupdate script.
>
> I'm adding a "failed-pre-rtupdate" hook for the next upload, which is
> run, if the pre-rtupdate hook fails (to allow the package to go to a
> sane state again).

that's a good idea obviously.


> I do have a different view on raising build and support information
> to the Packages/Sources files; in other cases the document clarifies
> things well.  IMO the current section 3.6 makes things more confusing
> than they are (at least for me).  Would it be helpful to add
> paragraphs starting with "Example:" in sections where they are
> useful? I.e. most package maintainers won't need the rtupdate
> scripts, and therefore could skip reading when they don't need these
> scripts.

I think the examples should be in an annex, that follow 3.6 TOC so that 
one can read the examples in front of the spec, without poluting the 
spec.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

Attachment: pgpB7Y3lWVpeT.pgp
Description: PGP signature


Reply to: