Bug#897238: Re: Bits about Intel MKL packaging -- Higher Priority than OpenBLAS
Just forgot to CC the RFS bug.
----- Forwarded message from Lumin <cdluminate@gmail.com> -----
Date: Sun, 6 May 2018 08:29:29 +0000
From: Lumin <cdluminate@gmail.com>
To: sebastien@debian.org
Cc: debian-science@lists.debian.org
Subject: Re: Re: Bits about Intel MKL packaging -- Higher Priority than OpenBLAS
User-Agent: Mutt/1.9.5 (2018-04-13)
Hello guys,
I've just updated the MKL packaging, and added a BLAS performance tester
there. I'll test the package soon, but let me first describe the
changes.
> He put forward a simpler solution: Just don't provide libblas.so.3, such
> that MKL will never be used to satisfy the dependency of libblas.so.3 .
> Based on his idea, my new solution is the follows:
>
> libmkl-rt --
> Depends: libblas3 | libblas.so.3
> Provides: NOTHING ... (4)
>
> So it's totally safe now. If there is MKL, there must be a free
> libblas.so.3 implementation with a priority definitely larger than 1,
> overriding MKL in terms of auto-mode alternatives. On the other hand,
> if that alternative is manually set, then there is nothing to worry
> about. This solution is also able to resolve problems found in (1) and (3).
Now libmkl-rt doesn't provide libblas.so.3 and liblapack.so.3, and it
pre-depends on libblas3 | libblas.so.3 and liblapack3 | liblapack.so.3 .
Similar change was applied to libmkl-dev.
> Apart from all things discussed above, there is upstream confirmation
> to the ambiguous license declaration in several headers. See [1]
Now the copyright file is updated. Copyright is clean.
> Thanks. There is a small bug: in both libmkl-rt.postinst.in and
> libmkl-dev.postinst.in, when the users reply "no", you put both BLAS and LAPACK
> alternatives in auto mode if MKL was selected for BLAS. You should rather split
> that in two tests: one for BLAS, one for LAPACK, because in theory it's
> possible to have BLAS pointing to MKL and not LAPACK.
One more debconf menu is added to configure script. A multiselect box
will be presented to the user so that we can treat BLAS and LAPACK
separately. That multiselect menu only appear once. For example, if the
user chose to point BLAS to MKL but not LAPACK, i.e.
libblas.so.3-x86_64-linux-gnu -> libmkl_rt.so
liblapack.so.3-x86_64-linux-gnu -> won't be touched
then following the configuration, these symlink will be set accordingly:
libblas.so.3-i386... -> libmkl_rt.so
liblapack.so.3-i386... -> won't be touched
libblas.so-{amd64,i386} -> libmkl_rt.so
liblapack.so-{amd64,i386} -> won't be touched.
For detail see [1][2][3]. Is this acceptable? If it's good, I'll test it
soon.
Previously Sébastien expressed his interest on benchmarking. I'm
interested in that too. So apart from the debconf part, I wrote a simple
benchmarker[4] in Julia[5] for comparing several BLAS implementations.
Thanks to Julia's convenient FFI functionality, I can test all
implementations within a single run, without any modification to environment
variable or alternatives.
Test result on my laptop (CPU=i5-7440HQ) matches expectation:
dnrm2 Julia
0.000033 seconds
dnrm2 /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.8.0
0.000074 seconds (3 allocations: 48 bytes)
dnrm2 Error :1.1368683772161603e-13
dnrm2 /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3
0.000038 seconds (3 allocations: 48 bytes)
dnrm2 Error :0.0
dnrm2 /usr/lib/x86_64-linux-gnu/libopenblas_haswellp-r0.2.20.so
0.000031 seconds (3 allocations: 48 bytes)
dnrm2 Error :0.0
dnrm2 /usr/lib/x86_64-linux-gnu/libmkl_rt.so
0.029561 seconds (3 allocations: 48 bytes)
dnrm2 Error :0.0
dgemm Julia
4.362279 seconds (2 allocations: 128.000 MiB, 0.20% gc time)
dgemm /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.8.0
47.893710 seconds (10 allocations: 160 bytes)
dgemm Error :2.0735139719127268e-10
dgemm /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3
10.412422 seconds (10 allocations: 160 bytes)
dgemm Error :2.4175670445887973e-11
dgemm /usr/lib/x86_64-linux-gnu/libopenblas_haswellp-r0.2.20.so
1.211220 seconds (10 allocations: 160 bytes)
dgemm Error :2.770610675980814e-11
dgemm /usr/lib/x86_64-linux-gnu/libmkl_rt.so
1.103356 seconds (10 allocations: 160 bytes)
dgemm Error :2.7982744719588258e-11
Netlib is always the slowest one. For small matrices OpenBLAS is
very competitive. For large matrices MKL is the fastest.
[1] https://salsa.debian.org/science-team/intel-mkl/blob/master/debian/libmkl-rt.templates
[2] https://salsa.debian.org/science-team/intel-mkl/blob/master/debian/libmkl-rt.config.in
[3] https://salsa.debian.org/science-team/intel-mkl/blob/master/debian/libmkl-rt.postinst.in
[4] https://salsa.debian.org/science-team/intel-mkl/blob/master/debian/tests/blascomp.jl
[5] http://julialang.org/
My Julia version is 0.6.2, which doesn't exist in Debian archive.
----- End forwarded message -----
Reply to: