Bug#857266: unblock: bottleneck/1.2.0-6
Package: release.debian.org
Severity: normal
User: release.debian.org@packages.debian.org
Usertags: unblock
Please unblock package bottleneck
The last upload of src:bottleneck which migrated to Stretch (1.2.0-4)
suffers regressions (including a segfault) with the latest release of
Numpy (1.12.x).
These were found later after the freeze deadline when I started
packaging src:python-xarray. Its test suite reveiled the regressions,
which were not caught by the upstream one. The regressions have now been
fixed after iterating with both upstream and Numpy.
I have cherry-picked the relevant fixes onto the packaging in unstable
(1.2.0-5, 1.2.0-6). Please consider applying the debdiff between 1.2.0-4
and 1.2.0-6 attached below.
Cheers,
Ghis
unblock bottleneck/1.2.0-6
-- System Information:
Debian Release: 9.0
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)
Kernel: Linux 4.9.0-2-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
diff -Nru bottleneck-1.2.0/debian/changelog bottleneck-1.2.0/debian/changelog
--- bottleneck-1.2.0/debian/changelog 2017-01-25 12:24:12.000000000 +0000
+++ bottleneck-1.2.0/debian/changelog 2017-02-26 13:58:26.000000000 +0000
@@ -1,3 +1,22 @@
+bottleneck (1.2.0-6) unstable; urgency=medium
+
+ * Cherry-pick upstream fix for segfaults happening while running the tests.
+ New patch 0003-Fix-segfault-caused-by-bad-refcount.patch (Closes: #856141)
+ * Run autopkgtests for all supported Python versions
+
+ -- Ghislain Antony Vaillant <ghisvail@gmail.com> Sun, 26 Feb 2017 13:58:26 +0000
+
+bottleneck (1.2.0-5) unstable; urgency=medium
+
+ [ Ghislain Antony Vaillant ]
+ * Cherry-pick upstream fix for Numpy 1.12
+ - New patch 0002-Fix-issue-with-relaxed-strides.patch
+
+ [ Pietro Battiston ]
+ * Added Ghislain Antony Vaillant as Uploader
+
+ -- Pietro Battiston <me@pietrobattiston.it> Tue, 31 Jan 2017 13:49:00 +0100
+
bottleneck (1.2.0-4) unstable; urgency=medium
[ Ghislain Antony Vaillant ]
diff -Nru bottleneck-1.2.0/debian/control bottleneck-1.2.0/debian/control
--- bottleneck-1.2.0/debian/control 2017-01-19 20:30:29.000000000 +0000
+++ bottleneck-1.2.0/debian/control 2017-02-07 07:39:59.000000000 +0000
@@ -1,5 +1,6 @@
Source: bottleneck
Maintainer: Pietro Battiston <me@pietrobattiston.it>
+Uploaders: Ghislain Antony Vaillant <ghisvail@gmail.com>
Section: python
Priority: optional
Build-Depends: dh-python, python-setuptools (>= 0.6b3), python3-setuptools,
diff -Nru bottleneck-1.2.0/debian/patches/0002-Fix-issue-with-relaxed-strides.patch bottleneck-1.2.0/debian/patches/0002-Fix-issue-with-relaxed-strides.patch
--- bottleneck-1.2.0/debian/patches/0002-Fix-issue-with-relaxed-strides.patch 1970-01-01 01:00:00.000000000 +0100
+++ bottleneck-1.2.0/debian/patches/0002-Fix-issue-with-relaxed-strides.patch 2017-02-07 07:39:59.000000000 +0000
@@ -0,0 +1,43 @@
+From: Keith Goodman <kwgoodman@gmail.com>
+Date: Sun, 29 Jan 2017 07:19:35 -0800
+Subject: Fix issue with relaxed strides
+
+---
+ bottleneck/src/iterators.h | 6 ++++--
+ bottleneck/tests/util.py | 1 +
+ 2 files changed, 5 insertions(+), 2 deletions(-)
+
+diff --git a/bottleneck/src/iterators.h b/bottleneck/src/iterators.h
+index 3ac68cb..5bb88d6 100644
+--- a/bottleneck/src/iterators.h
++++ b/bottleneck/src/iterators.h
+@@ -81,13 +81,15 @@ init_iter_all(iter *it, PyArrayObject *a, int ravel, int anyorder)
+ it->length = 1;
+ it->astride = 0;
+ }
+- else if (C_CONTIGUOUS(a)) {
++ /* The &&! in the next two else ifs is to deal with relaxed
++ * stride checking introduced in numpy 1.12.0; see gh #161 */
++ else if (C_CONTIGUOUS(a) && !F_CONTIGUOUS(a)) {
+ it->ndim_m2 = -1;
+ it->axis = ndim - 1;
+ it->length = PyArray_SIZE(a);
+ it->astride = strides[ndim - 1];
+ }
+- else if (F_CONTIGUOUS(a)) {
++ else if (F_CONTIGUOUS(a) && !C_CONTIGUOUS(a)) {
+ if (anyorder || !ravel) {
+ it->ndim_m2 = -1;
+ it->length = PyArray_SIZE(a);
+diff --git a/bottleneck/tests/util.py b/bottleneck/tests/util.py
+index 7f1f2a7..15e4238 100644
+--- a/bottleneck/tests/util.py
++++ b/bottleneck/tests/util.py
+@@ -155,6 +155,7 @@ def array_generator(func_name, dtypes):
+ yield a.reshape(shape)
+
+ # non-contiguous arrays
++ yield np.array([[1, 2], [3, 4]])[:, [1]] # gh 161
+ for dtype in dtypes:
+ # 1d
+ a = np.arange(12).astype(dtype)
diff -Nru bottleneck-1.2.0/debian/patches/0003-Fix-segfaults-caused-by-bad-refcount.patch bottleneck-1.2.0/debian/patches/0003-Fix-segfaults-caused-by-bad-refcount.patch
--- bottleneck-1.2.0/debian/patches/0003-Fix-segfaults-caused-by-bad-refcount.patch 1970-01-01 01:00:00.000000000 +0100
+++ bottleneck-1.2.0/debian/patches/0003-Fix-segfaults-caused-by-bad-refcount.patch 2017-02-26 13:58:26.000000000 +0000
@@ -0,0 +1,124 @@
+From: Keith Goodman <kwgoodman@gmail.com>
+Date: Thu, 9 Feb 2017 12:39:01 -0800
+Subject: Fix segfaults caused by bad refcount
+
+---
+ bottleneck/src/iterators.h | 11 +++++++++--
+ bottleneck/src/reduce_template.c | 12 ++++++++++++
+ 2 files changed, 21 insertions(+), 2 deletions(-)
+
+diff --git a/bottleneck/src/iterators.h b/bottleneck/src/iterators.h
+index 5bb88d6..563a823 100644
+--- a/bottleneck/src/iterators.h
++++ b/bottleneck/src/iterators.h
+@@ -21,6 +21,7 @@ struct _iter {
+ npy_intp astrides[NPY_MAXDIMS]; /* a.strides, a.strides[axis] removed */
+ npy_intp shape[NPY_MAXDIMS]; /* a.shape, a.shape[axis] removed */
+ char *pa; /* pointer to data corresponding to indices */
++ PyArrayObject *a_ravel; /* NULL or pointer to ravelled input array */
+ };
+ typedef struct _iter iter;
+
+@@ -59,6 +60,11 @@ init_iter_one(iter *it, PyArrayObject *a, int axis)
+ }
+ }
+
++/*
++ * If both ravel != 0 and it.a_ravel != NULL then you are responsible for
++ * calling Py_DECREF(it.a_ravel) after you are done with the iterator.
++ * See nanargmin for an example.
++ */
+ static BN_INLINE void
+ init_iter_all(iter *it, PyArrayObject *a, int ravel, int anyorder)
+ {
+@@ -70,6 +76,7 @@ init_iter_all(iter *it, PyArrayObject *a, int ravel, int anyorder)
+ it->axis = 0;
+ it->its = 0;
+ it->nits = 1;
++ it->a_ravel = NULL;
+
+ if (ndim == 1) {
+ it->ndim_m2 = -1;
+@@ -101,7 +108,7 @@ init_iter_all(iter *it, PyArrayObject *a, int ravel, int anyorder)
+ } else {
+ a = (PyArrayObject *)PyArray_Ravel(a, NPY_CORDER);
+ }
+- Py_DECREF(a);
++ it->a_ravel = a;
+ it->length = PyArray_DIM(a, 0);
+ it->astride = PyArray_STRIDE(a, 0);
+ }
+@@ -113,7 +120,7 @@ init_iter_all(iter *it, PyArrayObject *a, int ravel, int anyorder)
+ } else {
+ a = (PyArrayObject *)PyArray_Ravel(a, NPY_CORDER);
+ }
+- Py_DECREF(a);
++ it->a_ravel = a;
+ it->length = PyArray_DIM(a, 0);
+ it->astride = PyArray_STRIDE(a, 0);
+ }
+diff --git a/bottleneck/src/reduce_template.c b/bottleneck/src/reduce_template.c
+index 3d5183e..2bdf8dc 100644
+--- a/bottleneck/src/reduce_template.c
++++ b/bottleneck/src/reduce_template.c
+@@ -11,6 +11,12 @@
+ iter it; \
+ init_iter_all(&it, a, 1, 0);
+
++/* used with INIT_ALL_RAVEL */
++#define DECREF_INIT_ALL_RAVEL \
++ if (it.a_ravel != NULL) { \
++ Py_DECREF(it.a_ravel); \
++ }
++
+ #define INIT_ONE(dtype0, dtype1) \
+ iter it; \
+ PyObject *y; \
+@@ -545,6 +551,7 @@ REDUCE_ALL(NAME, DTYPE0)
+ Py_ssize_t idx = 0;
+ INIT_ALL_RAVEL
+ if (SIZE == 0) {
++ DECREF_INIT_ALL_RAVEL
+ VALUE_ERR("numpy.NAME raises on a.size==0 and axis=None; "
+ "So Bottleneck too.");
+ return NULL;
+@@ -559,6 +566,7 @@ REDUCE_ALL(NAME, DTYPE0)
+ }
+ }
+ BN_END_ALLOW_THREADS
++ DECREF_INIT_ALL_RAVEL
+ if (allnan) {
+ VALUE_ERR("All-NaN slice encountered");
+ return NULL;
+@@ -613,6 +621,7 @@ REDUCE_ALL(NAME, DTYPE0)
+ npy_DTYPE0 ai, extreme = BIG_INT;
+ INIT_ALL_RAVEL
+ if (SIZE == 0) {
++ DECREF_INIT_ALL_RAVEL
+ VALUE_ERR("numpy.NAME raises on a.size==0 and axis=None; "
+ "So Bottleneck too.");
+ return NULL;
+@@ -626,6 +635,7 @@ REDUCE_ALL(NAME, DTYPE0)
+ }
+ }
+ BN_END_ALLOW_THREADS
++ DECREF_INIT_ALL_RAVEL
+ return PyInt_FromLong(idx);
+ }
+
+@@ -844,6 +854,7 @@ REDUCE_ALL(NAME, DTYPE0)
+ done:
+ BUFFER_DELETE
+ BN_END_ALLOW_THREADS
++ DECREF_INIT_ALL_RAVEL
+ return PyFloat_FromDouble(med);
+ }
+
+@@ -888,6 +899,7 @@ REDUCE_ALL(median, DTYPE0)
+ BUFFER_DELETE
+ }
+ BN_END_ALLOW_THREADS
++ DECREF_INIT_ALL_RAVEL
+ return PyFloat_FromDouble(med);
+ }
+
diff -Nru bottleneck-1.2.0/debian/patches/series bottleneck-1.2.0/debian/patches/series
--- bottleneck-1.2.0/debian/patches/series 2017-01-19 20:30:29.000000000 +0000
+++ bottleneck-1.2.0/debian/patches/series 2017-02-26 13:58:26.000000000 +0000
@@ -1 +1,3 @@
0001_normalize_version.patch
+0002-Fix-issue-with-relaxed-strides.patch
+0003-Fix-segfaults-caused-by-bad-refcount.patch
diff -Nru bottleneck-1.2.0/debian/tests/control bottleneck-1.2.0/debian/tests/control
--- bottleneck-1.2.0/debian/tests/control 2017-01-22 13:08:05.000000000 +0000
+++ bottleneck-1.2.0/debian/tests/control 2017-02-26 13:58:26.000000000 +0000
@@ -1,9 +1,17 @@
-Test-Command: cd $AUTOPKGTEST_TMP
- ; python -c "import bottleneck as bn; bn.test()"
-Depends: python-bottleneck, python-nose
+Test-Command: set -e
+ ; for py in $(pyversions -r 2>/dev/null)
+ ; do cd "$AUTOPKGTEST_TMP"
+ ; echo "Testing with $py:"
+ ; $py -c "import bottleneck as bn; bn.test()"
+ ; done
+Depends: python-all, python-bottleneck, python-nose
Restrictions: allow-stderr
-Test-Command: cd $AUTOPKGTEST_TMP
- ; python3 -c "import bottleneck as bn; bn.test()"
-Depends: python3-bottleneck, python3-nose
+Test-Command: set -e
+ ; for py in $(py3versions -r 2>/dev/null)
+ ; do cd "$AUTOPKGTEST_TMP"
+ ; echo "Testing with $py:"
+ ; $py -c "import bottleneck as bn; bn.test()"
+ ; done
+Depends: python3-all, python3-bottleneck, python3-nose
Restrictions: allow-stderr
Reply to: