Challenge from Julia's non-standard vendored openblas"64_"

[Mo Zhou <lumin@debian.org>]

Hi developers,

A long existing historical problem that stems from
Fortran finally started to bite people.

--- Background

BLAS/LAPACK is a set of very stable API/ABIs for
dense numerical linear algebra, of "libc-level"
importance to scientific computing users. The
standard/reference/low-performance implementation
was written in fortran. Fortran compilers support
a weird feature to change the default INTEGER
length. Which means if you write a fortran
subroutine (in C-pseudo)

   sasum(INT N, float* X, INT INCX)

You'll get the sasum(int64_t, float*, int64_t)
function with a "-fdefault-integer-8" option.
The "INT" arguments in the above example are
used for 1-d array indexing.

The scientific computing community has got used
to this fortran feature, and many compatible
BLAS/LAPACK implementations support build flags
to select the "default integer size" even if
the implementation is written in C. The historical
problem is that the 64-bit indexing variant doesn't
have a standard SONAME.

--- Debian's WIP BLAS64/LAPACK64 libraries

I'm working on the 64bit-indexing support for
Debian's BLAS/LAPACK libraries. As discussed very
long time ago, we (mainly BLAS/LAPACK maintainers)
decided to use the "SONAME=libXXX64.so" convention
without mangling symbol names. Mangling is not
considered because only openblas supports it.
Actually, isolating the 32-bit and 64-bit indexing
variants with different SONAMES is expected to
be enough for avoiding confliction.

--- Julia's practice

To avoid symbol confliction with the 32bit-indexing
BLAS/LAPACK, Julia upstream changed the SONAME of
their vendored OpenBLAS to "libopenblas64_.so.*",
and **MANGLED** all the BLAS/LAPACK symbols by
appending the "64_" suffix. As a result, a portion
of the pre-built binaries they release are linked
against this libopenblas64_.so, and requiring
the mangled symbols (readelf -sW xxx). At the
same time the linux distribution Julia packages
built against 32bit version of 64-bit version
without symbol mangling will run into problem
as long as the user tries to install some
fundamental .jl packages with the Julia's
built-in package manager.

I have no confidence at all to convince
upstream to change their widely-spread practice.
If I

1. Patch Julia code and link it against the
   WIP BLAS64/LAPACK64 libraries (where there
   is no symbol mangling). Users will always
   have problem installing any external .jl
   packages.

2. Specifically compile a libopenblas64_.so
   from src:openblas for julia's use.
   This looks very bad for src:openblas's
   package layout.

3. Embed openblas source and let Julia's
   build system compile whatever it wants.
   In this case everything will be fine,
   except for the technical gracefulness.

I hesitate to choose a solution from them.
Advise?

---

P.S. This mail is again very long. But I'm afraid
that most people won't be able to understand the
problem if I only describe the core problem with
several simple words.