Re: Debian Python policy.

[Donovan Baarda <abo@minkirri.apana.org.au>]

Quoting Neil Schemenauer <nas@python.ca>:
[...]

I've done some digging in the archives and found things that look surprisingly 
like "my proposal" proposed by others. I don't think the finer points of how it 
would work were pinned down though, so I'm going to persist untill someone 
tells me to shut up  (sorry :-)

> Donovan Baarda wrote:
> > From archive updating point of view, my scheme has a large
> > python-X.Y-foo added and a small python-foo updated when python
> > upgrades. Your scheme has a large python-foo updated and a large
> > python-X.Y-foo added, where python-X.Y-foo is mostly the same as the
> > old python-foo.
> 
> By the time Python X.Y+1 comes out there is probably a micro release of
> X.Y available.

true, I guess, but are you going to bother? With version packages you can 
forget about the old X.Y release because the old packages will still all work 
fine. You can forget about a new micro release and focus on the new version.

> > That is, if you actualy intend to produce python-X.Y packages... I
> > suspect you wouldn't bother with your scheme.
> 
> Only if absolutely necessary.  Does Potato need Python 1.6 or 2.0?  I
> suspect not.  I'm not sure it even needs 1.5.2.

Maybe not right now, but 4 months ago it sure did. Every time Python does an 
upgrade, there is a transition time... it would be nice if all packages could 
magicly upgrade at once, but in practice it doesn't happen. It would be good if 
we could minimize the breakages during that transition.

> > Something that does complicate it all is even python modules that are
> version-
> > independant become tied to a particular version when they are
> installed 
> > into /usr/lib/python-X.Y (as in your python-bar above).
> 
> That's not as bad as progams becoming version dependent (as happens in
> your proposal).  So far I have about 70 modules on my list that need to
> be updated when python is updated.  There are over 100 packaged programs
> that use python.  From my initial investigation it looks like most of
> them will work fine if python becomes version 2.1.1.

My proposal doesn't make programs version dependant unless they already are; if 
they really require python-X.Y, they depend on python-X.Y and 
have "#!/usr/bin/env pythonX.Y". If they don't, they just depend on python and 
have "#!/usr/bin/env python". The python package depends on the latest python-
X.Y and provides a link /usr/bin/python to the latest /usr/bin/python-X.Y.

> > 3) have python-foo which depends on python (>=X.Y) install into
> > /usr/lib/python, and install links in /usr/lib/pythonA.B/ to
> > /usr/lib/python/* for all A.B >= X.Y. Note that these links will need
> > to be updated every time a new python-X.Y or python-foo is installed.
> > Perhaps a utility "update-links" function in the python (X.Y) package
> > should be provided to do this.
> > 
> > Note that 1) and 2) will break if a python upgrade changes the python
> > byte- code. Only 3) is bytecode-change-resistant.
> 
> That won't work for extension modules.  Also, bytecode is only
> compatible between micro releases.

Perhaps I didn't make it clear... some packages really are tied to a particular 
version of python, and some aren't. The above describes what do do for packages 
that don't depend on a particular version.

Packages like extension modules _are_ tied to a particular version and hence 
should be in a python-X.Y-foo package that installs into /usr/lib/pythonX.Y. 
There would also be an empty package python-foo that simply depends on the 
latest python-X.Y-foo and python packages.

examples;

python is at version 1.5.
package python-1.5 (1.5.2-1) installs into /usr/lib/python1.5 and updates links 
from /usr/lib/python/* to /usr/lib/python1.5, compiling them to *.pyc files.
package python (1.5.2-1) depends on python-1.5 and provides a 
link /usr/bin/python to /usr/bin/python1.5

python-spam is a compiled extension at version 3.2.
It is tied to a particular version of python and hence needs versioned packages.
package python-1.5-spam (3.2-1) depends on python-1.5 and installs 
into /usr/lib/python1.5
package python-spam (3.2-1) depends on python-1.5-spam and python (>=1.5,<1.6)

python-eggs is a python module at version 2.1
It is not tied to a particular version and hence doesn't need versioned 
packages.
package python-eggs (2.1-2) depends on python (>=1.5.2) and installs 
into /usr/lib/python. It also installs symlinks in all /usr/lib/pythonM.N/ 
directories to it's *.py files and compiles them all using the appropriate 
pythonX.Y

foo is a program that embeds python 1.5 at version 5.3 and uses spam and eggs.
It is tied to a particular version of python, hence depends on the versioned 
packages.
package foo (5.3-1) depends on python-1.5, python-1.5-spam, and python-eggs 
(>=2.1). Any scripts it has use "#!/usr/bin/env python1.5"

bar is a python program at version 3.2 that uses spam and eggs.
It is not tied to a particular version of python, hence depends on the 
unversioned packages.
package bar (3.2-1) depends on python (>=1.5.2), python-spam (>=3.2), and 
python-eggs (>=2.1). Any scripts it has use "#!/usr/bin/env python"

python 2.1 is released
new package python-2.1 (2.1.0-1) installs into /usr/lib/python2.1 and updates 
and compiles links from /usr/lib/python to /usr/lib/python2.1 (including python-
eggs).
updated package python (2.1.0-1) depends on python-2.1 and updates the link 
from /usr/bin/python to /usr/bin/python2.1

At this point the following packages are broken because they need to be updated 
for the new python; python-spam (needs python >=1.5,<1.6), and bar (needs 
python-spam). However, all of the rest still works, including python-eggs for 
both versions of python.

To fix bar, a new version of spam compiled against python-2.1 is needed.
new package python-2.1-spam (3.2-1) depends on python-2.1 and installs 
into /usr/lib/python2.1
updated package python-spam (3.2-2) depends on python-2.1-spam and python 
(>=2.1,<2.2)

Now everything works again. bar has not been updated, but happily runs with the 
new python 2.1. foo has not been updated but still runs using python 1.5.2 and 
python-1.5-spam. python-eggs has not been updated but works with all versions 
of python. There is now two versions of python-spam to support the two 
different versions of python. 

A new version of python has been installed with all breakages for 2 programs 
and 2 extensions resolved by creating 2 new packages, and updating 2 small 
wrapper packages. Your proposal would require 5 packages all updated to the 
latest python. If foo could not be updated to python 2.1 for whatever reason, 
it would require 5 packages updated, and 2 additional versioned packages.

The gist of the idea is all python components have un-versioned packages. Some 
components are version specific, and hence have versioned packages and the 
unversioned package acts as a loose wrapper to pull in the latest versioned 
package. Components and programs that are version specific should depend on the 
versioned packages. Components and programs that are not version specific 
should depend on the unversioned packages.

The only thing that can catch you with this is new versions of python breaking 
supposedly version independant components. If you have some premonition that 
this is going to happen (Deprecation) you can pre-empt it a bit by specifying 
an upper version limit in their dependancies (ie python-eggs depends on python 
>=1.5.2, <2.2), but all this does is ensure the package breaks when python is 
up-reved. I'm not sure how to fix this neatly, but I suspect it would require 
some package jiggery in any situation.

--
ABO: finger abo@minkirri.apana.org.au for more information.