[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Tiledb-py fails to build (Was: tiledb: uses atomic operations, but is not linked to libatomic)



Hello,

It seems unittests are running against very new pandas version that wants pyarrow (but it does not seems to be available in the archive).
Can you check while self.use_arrow evaluate to True ? That's mostly the root cause of the failure.
I'll try to have a look when I'll be in front of my computer.

Regards, Adam.

On March 25, 2023 2:55:39 PM GMT+01:00, Andreas Tille <andreas@an3as.eu> wrote:
Hi,

as you can read in the bug log, there was an upload of a new version of
tiledb a couple of hours before it has migrated to testing. Thus the
package remains affected by a testing removal (together with its two
reverse dependencies tiledb and genomicsdb). To follow the freeze
policy I reverted the version bump and NMUed tiledb
2.15.0really2.14.1-0.1 to experimental (since the maintainer did not
responded).

As we can see tiledb-py does not build against tiledb 2.15.0[1]

I've now forced (Build-)Depends to
tibtiledb-dev (>= 2.15.0really2.14.1~)
but it seems Salsa CI autopkgtest does not respect the setting

variables:
# Build against tiledb in experimental
RELEASE: 'experimental'

and thus the autopkgtest log does not reproduce the error I've got
in my local build:

...

=================================== FAILURES ===================================
_______________ TestNumpyToArray.test_from_numpy_empty_str[1-0] ________________

self = <tests.test_libtiledb.TestNumpyToArray object at 0x7fe177722710>
empty_str = '', num_strs = 1

@pytest.mark.parametrize("empty_str", ["", b""])
@pytest.mark.parametrize("num_strs", [1, 1000])
def test_from_numpy_empty_str(self, empty_str, num_strs):
uri = self.path("test_from_numpy_empty_str")
np_array = np.asarray([empty_str] * num_strs, dtype="O")
tiledb.from_numpy(uri, np_array)
....
with tiledb.open(uri, "r") as A:
assert_array_equal(A[:], np_array)
if has_pandas():
assert_array_equal(A.query(use_arrow=True).df[:][""], np_array)

tests/test_libtiledb.py:3356:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _.
/usr/lib/python3/dist-packages/tiledb/multirange_indexing.py:192: in __getitem__
return self if self.return_incomplete else self._run_query()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _.

self = <tiledb.multirange_indexing.DataFrameIndexer object at 0x7fe1779ef150>

def _run_query(self) -> Union[DataFrame, Table]:
if self.pyquery is not None:
self.pyquery.submit()
....
if self.pyquery is None:
df = DataFrame(self._empty_results)
elif self.use_arrow:
with timing("buffer_conversion_time"):
table = self.pyquery._buffers_to_pa_table()
E ModuleNotFoundError: No module named 'pyarrow'

/usr/lib/python3/dist-packages/tiledb/multirange_indexing.py:329: ModuleNotFoundError
_______________ TestNumpyToArray.test_from_numpy_empty_str[1-1] ________________

self = <tests.test_libtiledb.TestNumpyToArray object at 0x7fe177722490>
empty_str = b'', num_strs = 1

@pytest.mark.parametrize("empty_str", ["", b""])
@pytest.mark.parametrize("num_strs", [1, 1000])
def test_from_numpy_empty_str(self, empty_str, num_strs):
uri = self.path("test_from_numpy_empty_str")
np_array = np.asarray([empty_str] * num_strs, dtype="O")
tiledb.from_numpy(uri, np_array)
....
with tiledb.open(uri, "r") as A:
assert_array_equal(A[:], np_array)
if has_pandas():
assert_array_equal(A.query(use_arrow=True).df[:][""], np_array)

tests/test_libtiledb.py:3356:
/usr/lib/python3/dist-packages/tiledb/multirange_indexing.py:192: in __getitem__
return self if self.return_incomplete else self._run_query()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _.

self = <tiledb.multirange_indexing.DataFrameIndexer object at 0x7fe177a16ed0>

variables:
# Build against tiledb in experimental
RELEASE: 'experimental'
def _run_query(self) -> Union[DataFrame, Table]:
if self.pyquery is not None:
self.pyquery.submit()
....
if self.pyquery is None:
df = DataFrame(self._empty_results)
elif self.use_arrow:
with timing("buffer_conversion_time"):
table = self.pyquery._buffers_to_pa_table()
E ModuleNotFoundError: No module named 'pyarrow'

/usr/lib/python3/dist-packages/tiledb/multirange_indexing.py:329: ModuleNotFoundError
______________ TestNumpyToArray.test_from_numpy_empty_str[1000-0] ______________

self = <tests.test_libtiledb.TestNumpyToArray object at 0x7fe177722510>
empty_str = '', num_strs = 1000

@pytest.mark.parametrize("empty_str", ["", b""])
@pytest.mark.parametrize("num_strs", [1, 1000])
def test_from_numpy_empty_str(self, empty_str, num_strs):
uri = self.path("test_from_numpy_empty_str")
np_array = np.asarray([empty_str] * num

=================================== FAILURES ===================================
_______________ TestNumpyToArray.test_from_numpy_empty_str[1-0] ________________

self = <tests.test_libtiledb.TestNumpyToArray object at 0x7fe177722710>
empty_str = '', num_strs = 1

@pytest.mark.parametrize("empty_str", ["", b""])
@pytest.mark.parametrize("num_strs", [1, 1000])
def test_from_numpy_empty_str(self, empty_str, num_strs):
uri = self.path("test_from_numpy_empty_str")
np_array = np.asarray([empty_str] * num_strs, dtype="O")
tiledb.from_numpy(uri, np_array)
....
with tiledb.open(uri, "r") as A:
assert_array_equal(A[:], np_array)
if has_pandas():
assert_array_equal(A.query(use_arrow=True).df[:][""], np_array)

tests/test_libtiledb.py:3356:
...


I admit I'm a bit astonished since when tiledb 2.14.1 was available
in unstable the autopkgtest used to work. Any idea how to fix this
issue would be welcome.

Kind regards,
Andreas

[1] https://salsa.debian.org/python-team/packages/tiledb-py/-/pipelines/514328


Reply to: