[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: RFC: threadding-aware virtual BLAS/LAPACK?

On Thu, 7 May 2020 at 00:49, Mo Zhou <lumin@debian.org> wrote:

Through some previous discussions I realized that the issue of
threadding implementation of the BLAS can be sometimes cumbersome.  For
example, sometimes we can observe severe performance regression from
pthread program + BLAS with gomp, or even observe severe calculation
error from gomp program + BLAS with iomp (aka llvm openmp).

These threadding disasters will be eventually propagated to our end
users, and may possibly harm their scientific computing experience.

This potential disaster has already occurred:

Installing R-4.0.0 on Debian and Ubuntu resulted in systems hanging
on simple matrix operations.   This was discussed on the R-sig-debian
list.  Quoting a message from Samuele Carcagno:

... after some further investigation I found out that the issue was related
to some calls to the `solve` function in `make.rgb`, and indeed running
`example(solve)` would hang my R session. After I changed the default
BLAS with:

sudo update-alternatives --config libblas.so.3-x86_64-linux-gnu

the problem disappeared. The BLAS output of `sessionInfo` now that
`solve` is working is:

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3

while before when `solve` was not working it was:

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3

so it seems the issue is with the Ubuntu package `libopenblas0-pthread`
and has nothing to do with R. I don't know why openblas-pthread was the
default BLAS on my system.

Fedora uses libRblas private to R.  The R documentation notes that many of the faster
implementations may give different results.   

------------- Fedora

Recall what Fedora does for the BLAS libraries: they dont do any virtual
package at all. OpenBLAS packages with different threadding support are
given different sonames:

  openblas + pthread: libopenblasp.so.*
  openblas + openmp:  libopenblaso.so.* (IIRC)
  openblas + serial:  libopenblass.so.* (IIRC)

Although this makes implementations not switchable at runtime, but at
least the users won't have to struggle with threads.

------------- Gentoo

I wrote the alternatives mechanism for gentoo's blas/lapack, which
resembles Debian's. However, gentoo's package management system supports
a "USE flag" feature which allows the user to set global threadding
implementation for the whole system.

This if fine for a system that is dedicated to a single workflow, but
may not be good for a desktop workstation used for a range of
calculations with different packages (including user's own 
code) where most software can use a portable BLAS but one
calculation needs optimizations.

------------- what can we do

Maybe we can provide some more virtual shared objects such as
libblasp.so, which has candidates such as openblas-pthread and

In that case if a Debian maintainer intentionally choose to link against
a pthread BLAS (libblasp) to avoid issues of an openmp BLAS (libblaso),
we can help the users to avoid threadding issues when they don't read
any documentation (actually I think 99% of the users won't read the doc,
or have enough background to understand the threadding issue).

We can dig deeper into this direction if this turns to be a sensible
direction for making improvement.

The priority for Debian should be for widely used packages like 
R to be consistent with other distros, which means R should
provide libRblas so the 95% of users will not encounter 
problems using their distro's packages.   Some users may
have use cases that need alternative BLAS, and will likely
end up building their own configuration.  There are advantages
to keeping libraries consistent with other distros so we don't
end up with distro-specific build instructions for widely used

This is a part of my GSoC2020 project, although this topic is not on the
original plan.

George N. White III

Reply to: