[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Numpy API change?



Hi all,

On Tue, Jul 20, 2010 at 15:07, Jakub Wilk <jwilk@debian.org> wrote:
> * Antoine Sirinelli <antoine@monte-stello.com>, 2010-07-19, 21:27:
>>
>> It seems that since the last numpy upload, pytables is no longer working.
>> I
>> have filled a bug report
>> (http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=589669) but my question
>> is
>> more general. It seems that pytables is broken because of a change in
>> numpy
>> API. Is there a way to avoid that using good dependences in the different
>> deb
>> packages?
>>
>> I may be completely wrong about the origin of this bug. Sorry if it is the
>> case.
>
> Size of numpy.dtype has grown in Numpy 1.4 (some new fields has been
> appended to the end of the structure). This itself does not constitute an
> ABI breakage; however, some Python extensions[1] are checking if size of
> imported types is exactly the same as it were at build time. Therefore, they
> won't work if built with Numpy 1.3 and run with Numpy 1.4 (or the other way
> round).
>
> Some other extensions[2] are more lax when it comes to comparing type sizes,
> and they crash only if the size shrank since build time.
>
> [1] At least these:
> - python-tables
> - mayavi2
> Rebuilding their Pyrex/Cython sources with recent Cython should move them
> into the second group.
>
> [2] At least these:
> - python-nipy-lib
> - python-petsc4py - python-scikits-learn-lib
> - python-scipy

We just had a chat on #debian-python in order to find a way to avoid
this situation to happen again in the future.

Fact is, we should find a way to tight the dependencies on
python-numpy, for example we can re-inforce some sort of "python-numpt
(>= MAJOR.MINOR), python-numpy (<< MAJOR.MINOR+1)" (where MAJOR is the
current major version number, and MINOR is the minor one). That means
that at each update of major/minor release of numpy, the depending
packages needs to be binNMU'ed or receive a sourceful upload.

We also need to find some way to automate this versioned information
addition and the following depends on that by depending packages.

Piotr developed a nice approach when writing dh_python2 (and promised
to backport it to python-support too): he introduced the pydist
concept. This is a way to express in the current package A what are
the dependencies other packages B, C, ... should use when depending on
A.

(the example is not strictly what will be in reality, it's just a
mock) Let's say python-a declares a pydist of "python-a << (x.y)",
then python-b can call the helper like "dh_python2 --with-pydist=A"
and have the dependencies on "python-a << (x.y)" automatically added
to its Depends line.

Now, this is very nice, but it's still not a solution we can use it
right now. Instead, what we can do is to prepare the ground to make
this happen when we're ready. So what I think we can do right now is:

- for the time being, declare a (build-)depends on python-numpy >=
1.4.1 in rdeps of python-numpy
- move any debian packages depending on python-numpy to use
python-support; if you want help to do it (either be some basic
guidelines or a complete&working patch), just contact me: I already
moved several package from central to support, so I know how to do
that and I'd be happy to help fixing part of this mess (that
incidentally I generated uploading 1.4.1 :) ).

Please express any thoughts or dubts, I'd be happy to hear them and
answer if I can :)

If I missed some packages in the loop, please forward it to them too.

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi


Reply to: