[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#920276: RFP: libopenpmd-api -- API for scientific I/O with openPMD

Package: wnpp
Severity: wishlist

* Package name    : openpmd-api
  Version         : 0.7.1
  Upstream Author : Axel Huebl <axel.huebl@plasma.ninja>
* URL             : https://openpmd-api.readthedocs.io
* Upstream Repo   : https://github.com/openPMD/openPMD-api
* License         : LGPLv3+
  Programming Lang: C++, Python
  Description     : API for scientific I/O with openPMD

Proposed Package Description

This library provides a high-level API for writing and reading scientific data according to openPMD. It allows to build logical file structures which drive scientific I/O libraries such as HDF5 and ADIOS through a common, intuitive interface. Where supported, openPMD-api implements both serial and MPI parallel I/O capabilities.

Why is this Package Useful/Relevant

openPMD is a meta-standard in scientific computing, allow seamless data exchange accross scientific and engineering domains. openPMD-api is a file-format agnostic (to the extend of its backends) API that allows to re-use the same data processing toolchains on various hierarchical representations, such as HDF5 files, JSON files, ADIOS files and file-less staging methods.

This library builds by default a C++ library and a Python3 library, whose artifacts we can separate in packaging.

Additional Relevant Information

I am the upstream maintainer of this software and our community currently uses the software actively with user-level package managers.

This software comes with and without MPI support. I would therefore propose to build three variants of it, similar as the HDF5 packages:
- variant 1: without MPI (libopenpmd-api-dev)
- variant 2: with OpenMPI (libopenpmd-api-openmpi-dev)
- variant 3: with MPICH (libopenpmd-api-mpich-dev)

It requires the following runtime dependencies (already packaged):
- variant 1: libhdf5-serial libadios python3
- variant 2: libhdf5-openmpi libadios-openmpi python3
- variant 3: libhdf5-mpich libadios-mpich python3

-dev variants of the above packages and python3-dev are needed at build time.

We can separate the build artifacts in ${BUILD_DIR}/lib/python*/, which contain a single CPython .so module, to provide a `python3-openpmd-api` package, depending on the C++ library (any variant). MPI-variants for those python bindings are not yet created and will be added in future releases (based on mpi4py). At runtime, the python3-openpmd-api package will require python3-numpy in version 1.15.0 or newer.

openPMD-api is designed in a way, that downstream packages that require only the serial functionality (variant 1) will also work if variant 2 or variant 3 is present (but not vice-versa).

openPMD-api has the following build-time "tool" dependencies:
- CMake 3.11.0+
- pkgconfig
- Makefiles or Ninja
- a C++11 capable compiler and its C variant (e.g. gcc & g++ v4.8+)

Since the CMake scripts of openPMD-api perform dependency auto-detection by default, it is advisable to fix dependencies during build time, to notice if something is missing. Also, please specify the python executable explicitly.

Here is an example build line installing into /usr, please perform as out-of-source build:

cmake \
  -DopenPMD_USE_JSON=ON    \
  -DopenPMD_USE_HDF=ON     \
  -DopenPMD_USE_ADIOS1=ON  \
  -DPYTHON_EXECUTABLE=$(which python3) \

make test
make install

If you want to set the C++ standard consequently throughout all packages, please don't just set CXXFLAGS but rather pass:


otherwise, the defaults for the current compiler will be taken.

Detailed build options:
- https://openpmd-api.readthedocs.io/en/latest/dev/buildoptions.html

Detailed dependencies:
- https://openpmd-api.readthedocs.io/en/latest/dev/dependencies.html

Please don't hesitate to contact me with further questions or by opening issues
on the above linked repo.

Thanks a lot for your help packaging openPMD-api!

Best regards,
Axel Huebl

Reply to: