Re: Joining ROCm team, ectrans
On 14/10/2024 09:51, Cordell Bloor wrote:
Hi Alastair,
On 2024-10-12 02:07, Alastair McKinstry wrote:
I plan to enable ROCm support for some packages used in weather and
climate modelling; in particular ectrans.
Are there good examples and docs on how ROCm works in Debian
available yet?
Upon reviewing ectrans, it looks like it is pretty straightforward.
You will need to add hipcc, libhipblas-dev, librocblas-dev,
libhipfft-dev, and librocfft-dev to the B-D, set HIPCXX=clang++-17 as
an environment variable, and specify which GPU architectures you're
building for by passing -DCMAKE_HIP_ARCHITECTURES=<arch> to your CMake
configure command.
I'd suggest
-DCMAKE_HIP_ARCHITECTURES="gfx803;gfx900;gfx906;gfx908;gfx90a;gfx1010;gfx1030;gfx1100;gfx1101;gfx1102"
to match the math libraries that ectrans depends on. We could also use
your help in getting HIPCXXFLAGS set up in dpkg-dev [2]. Until support
for the HIP language exists in dpkg-dev, you might have to copy flags
from CXXFLAGS to HIPCXXFLAGS when using CMake's HIP language support.
Alternatively, you can use the old-style method for building HIP
programs that uses clang or hipcc as the CXX compiler. This requires
all the same B-D, but you set CXX=clang++-17 or CXX=hipcc as an
environment variable. Typically, you specify the GPU architectures
you're building for in this mode by passing -DAMDGPU_TARGETS=<arch>,
but it appears that ectrans will use CMAKE_HIP_ARCHITECTURES instead.
If there are build flags that cause problems for GPU code, you can
prepend -Xarch_host to them in d/rules, like was done for rocSPARSE [2].
Once you've successfully enabled ROCm support in a library, perhaps
you could write the docs on this topic. :9
Thats impressive :-)
There are a number of related packages in the "European weather software
stack" that I maintain in Debian and work on professionally.
Most directly, "field-api" (https://github.com/ecmwf-ifs) is an API for
automatically copying data between CPUs and GPUs. Loki
(https://tracker.debian.org/pkg/loki-ecmwf) is a DSL for automatically
generating optimal CPU/GPU code from a Fortran code - write your weather
code to API guidelines and Loki will generate the GPU code.
This then uses several libraries (libfiat0d for Fortran support etc).
Some of this may be used/tested on production on large scale HPC systems
via. Docker (Singularity/Apptainer ) images. I am working on an AMD port
of a large weather code.
Sincerely,
Cory Bloor
Regards
Alastair
[1]:
https://github.com/ecmwf-ifs/ectrans/blob/1.5.0/cmake/ectrans_find_hip.cmake
[2]: https://bugs.debian.org/1065439
[3]:
https://salsa.debian.org/rocm-team/rocsparse/-/blob/debian/5.7.1-5/debian/rules?ref_type=tags#L7-9
--
Alastair McKinstry,
GPG: 82383CE9165B347C787081A2CBE6BB4E5D9AD3A5
e: alastair@mckinstry.ie, im: @alastair:mckinstry.ie @amckinstry@mastodon.ie
Commander Vimes didn’t like the phrase “The innocent have nothing to fear,”
believing the innocent had everything to fear, mostly from the guilty but in the longer term
even more from those who say things like “The innocent have nothing to fear.”
- T. Pratchett, Snuff
Reply to: