Re: numpy boolean subtract, the `-` operator, is deprecated, use the bitwise_xor, the `^` operator, or the logical_xor function instead (Was: Bug#899205: python-cogent: Test suite fails with latest matplotlib)

[Stuart Prescott <stuart@debian.org>]

Hi Andreas,

> ======================================================================
> ERROR: test_consistent_gap_degen_handling
> (test_core.test_sequence.ModelSequenceTests) gap degen character should be
> treated consistently
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
>   "/tmp/autopkgtest-
lxc.5a99fnj6/downtmp/autopkgtest_tmp/tests/test_core/test_sequence.py",
>   line 660, in test_consistent_gap_degen_handling
>     self.assertEqual(dna.stripBadAndGaps(), raw_ungapped)
>   File "/usr/lib/python2.7/dist-packages/cogent/core/sequence.py", line
>   1251, in stripBadAndGaps
>     valid_indices -= self._data == i
> TypeError: numpy boolean subtract, the `-` operator, is deprecated, use
> the bitwise_xor, the `^` operator, or the logical_xor function instead.
> 
> ======================================================================

The error message basically describes the solution -- patch attached. Quite 
what the original meaning of "bool - bool" was supposed to be is hard to 
discern (and that is why numpy has deprecated it). You should get upstream 
to look at this carefully.

Note that there is more than one test that fails like this. There also may 
be other places depending on how good their test coverage is.

There are also failing doctests; a patch for them also attached, permitting 
the test to work with both old and new numpy.

These two patches only get the package to compile though. Lintian will then 
tell you about other RC bugs that you need to fix prior to upload.

regards
Stuart


-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart@nanonanonano.net
Debian Developer   http://www.debian.org/         stuart@debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

Description: Use xor rather than subtraction on booleans
 Numpy no longer accepts the ambiguous construction of a-b when a and b are bools.
 Normally, this can be replaced with XORs.
Author: Stuart Prescott <stuart@debian.org>
--- a/cogent/core/sequence.py
+++ b/cogent/core/sequence.py
@@ -1248,7 +1248,7 @@
         gap_indices = map(self.Alphabet.index, self.MolType.Gaps)
         valid_indices = self._data < len(self.Alphabet)
         for i in gap_indices:
-            valid_indices -= self._data == i
+            valid_indices ^= self._data == i
         result = compress(valid_indices, self._data)
         return self.__class__(result, Info=self.Info)
 
--- a/cogent/maths/distance_transform.py
+++ b/cogent/maths/distance_transform.py
@@ -670,9 +670,9 @@
         return zeros((0,0),'d')
     dists = zeros((numrows,numrows),'d')
     for i in range(numrows):
-        r1 = datamtx[i] # cache here
+        r1 = datamtx[i].astype(dists.dtype) # cache here
         for j in range(i):
-            dists[i,j] = dists[j,i] = sum(abs(r1 - datamtx[j]))
+            dists[i,j] = dists[j,i] = sum(abs(r1 - datamtx[j].astype(dists.dtype)))
             
     return dists
 

Description: Ensure doctests pass with different numpy versions
 The numpy output format changed with numpy 1.14 and this causes doctests to
 fail if raw numerical output is compared. See
 https://wiki.debian.org/ContinuousIntegration/TriagingTips/numpy-1.14-doctests
Author: Stuart Prescott <stuart@debian.org>
--- a/tests/alltests.py
+++ b/tests/alltests.py
@@ -6,6 +6,14 @@
 import doctest, cogent.util.unit_test as unittest, sys, os
 from cogent.util.misc import app_path
 
+# Whitespace changes between numpy 1.13 and 1.14 will cause the doctests
+# to fail; when doctests are updated to 1.14 format, this can be removed.
+try:    # CRUFT
+    import numpy as np
+    np.set_printoptions(legacy='1.13')
+except TypeError:
+    pass
+
 __author__ = "Peter Maxwell and Gavin Huttley"
 __copyright__ = "Copyright 2007-2016, The Cogent Project"
 __credits__ = ["Peter Maxwell", "Gavin Huttley", "Rob Knight",