I am packaging apache tvm. It builds a C-library libtvm.so (and libtvm_runtime.so).
It also has a python interface which is how most people use it, so I've built that into python3-tvm
It has a /usr/bin/tvmc which fails if you run it due to not being able to find the installed c-libraries.
I have no idea how the python c-library-finding mechanism constructs
its path list, so I'm not sure where to prod this to make it
work. There is presumably a right place to add a path to look on, or
maybe to enable the 'it's in the standard debian system path - just do what /etc/ld.so.conf.d/* says'
functionality.
Currently I get this:
$ tvmc
Traceback (most recent call last):
File "/usr/bin/tvmc", line 33, in <module>
sys.exit(load_entry_point('tvm==0.8.0', 'console_scripts', 'tvmc')())
File "/usr/bin/tvmc", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/usr/lib/python3/dist-packages/tvm/__init__.py", line 26, in <module>
from ._ffi.base import TVMError, __version__, _RUNTIME_ONLY
File "/usr/lib/python3/dist-packages/tvm/_ffi/__init__.py", line 28, in <module>
from .base import register_error
File "/usr/lib/python3/dist-packages/tvm/_ffi/base.py", line 71, in <module>
_LIB, _LIB_NAME = _load_lib()
File "/usr/lib/python3/dist-packages/tvm/_ffi/base.py", line 51, in _load_lib
lib_path = libinfo.find_lib_path()
File "/usr/lib/python3/dist-packages/tvm/_ffi/libinfo.py", line 146, in find_lib_path
raise RuntimeError(message)
RuntimeError: Cannot find the files.
List of candidates:
/home/wookey/bin/libtvm.so
/usr/local/bin/libtvm.so
/usr/bin/libtvm.so
/bin/libtvm.so
/usr/local/games/libtvm.so
/usr/games/libtvm.so
/usr/lib/python3/dist-packages/tvm/libtvm.so
/usr/lib/libtvm.so
/home/wookey/bin/libtvm_runtime.so
/usr/local/bin/libtvm_runtime.so
/usr/bin/libtvm_runtime.so
/bin/libtvm_runtime.so
/usr/local/games/libtvm_runtime.so
/usr/games/libtvm_runtime.so
/usr/lib/python3/dist-packages/tvm/libtvm_runtime.so
/usr/lib/libtvm_runtime.so
So it tries quite hard to find it, but doesn't know about multiarch and thus fails to look in the right place:
/usr/lib/<triplet>/ (/usr/lib/x86_64-linux-gnu/ on this box)
Also does python really think that /usr/local/games/ should be cheked before /usr/lib/ ? That just seems wrong.
Clues about where to prod gratefully received.
I see that /usr/lib/python3/dist-packages/tvm/_ffi/libinfo.py contains
a function 'get_dll_directories' which looks promising and adds
TVM_LIBRARY_PATH to the search list and if I run tvmc like this:
TVM_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/ tvmc
then that path is at the top of the list.
OK, but that mostly reveals a second issue: it's looking for
libtvm.so, but that unversioned link is only provoded in the dev package
libtvm-dev. The library package has the versioned filenames
/usr/lib/x86_64-linux-gnu/libtvm.so.0
/usr/lib/x86_64-linux-gnu/libtvm_runtime.so.0
So I also have to persuade it to look for libtvm.so.0 not
libtvm.so. Where does that info live? OK, a bit more research shows
that that is in /usr/lib/python3/dist-packages/tvm/_ffi/libinfo.py
which is in the source as python/tvm/_ffi_libinfo.py, in find_lib_path
and that's easy to fix, and probably even the right place to fix it?
The paths is harder though. get_dll_directories in
python/tvm/_ffi_libinfo.py adds $PATH after $LD_LIBRARY_PATH to make
it's search list. Is searching $PATH for libraries ever right?
What it should actually be adding is what's in /etc/ld.so.conf.d/*
That can be listed with
/sbin/ldconfig -v 2>/dev/null | grep -v ^$'\t' | cut -d: -f1
(yuk? is there really no better way?)
How does one do that in python to get that set of path added in the
libinfo.py function?
https://github.com/apache/tvm/blob/main/python/tvm/_ffi/libinfo.py
Am I barking up the right tree here or is there a better way?
Wookey
--
Principal hats: Debian, Wookware, ARM
http://wookware.org/
Attachment:
signature.asc
Description: PGP signature