Re: CMake musings for ROCm 4.5.2 (Was Re: ROCm packaging session memo)
Thanks for the CMake discussion, Maxime. People sometimes wonder why I
spend so much time on the build system when there's far more interesting
problems I could be working on. It's messages like yours that help me
explain why it's worth spending time on our build.
On 2022-01-28 17:23, Maxime Chambonnet wrote:
On the right, you can see rocBLAS as it is now.
First, in library/CMakeLists.txt, GNUInstallDirs is included and its
variables CMAKE_INSTALL_{BINDIR, LIBDIR, INCLUDEDIR} (there exists
others [3]) are referred-to, which seems nice. But in the children
CMakeLists in library/src, which handles the actual install, the
aforementioned GNUInstallDirs are crippled and dropped for a mix of
CMAKE_SOURCE_DIR and hard-coded paths. Installing a package in
/usr/<package> is considered poor practice in Debian-based distribs.
It doesn't work properly because it wasn't designed to do that in ROCm
4.5. It's only just now that support for any layout besides that of
/opt/rocm and spack has become a goal. Upstream support probably won't
be coming until ROCm 5.2 or later, but we should be able to get it fixed
on Debian long before then by backporting patches (once they exist).
I believe I mentioned in the past that you can correct this problem
across all math libraries with minor changes to rocm-cmake. I've
recently been made a maintainer for rocm-cmake, so I suppose it's now my
job to help facilitate that. The primary question is what file layout
Debian wants.
I can imagine this nesting under /usr/rocrand + symlinking to /usr
classical sub-dirs, an operation being a remnant of the day that
/opt/rocm was the main assumption?
/usr/rocrand was the result of my quick-and-dirty sample build script.
It's not a serious suggestion of anything. It's a prompt for us to
discuss what layout Debian would like for the math libraries. For
example, is there a preference between /usr/include/rocrand/rocrand.h
and /usr/include/rocrand.h?
As I see it today, rocm-cmake is a gross over-complexity. I must be
short-sighted and you might have good reasons, (on top of the CPackDeb
infrastructure which has root in this repo) such as installations on
Windows, or maybe other Linux distros, for its existence?
I can imagine that much cmake could be cleaned if this was the case!
I have not dug deep enough into rocm-cmake to have a personal opinion.
It was authored by a skilled developer, so I default to trusting his
judgement.
In general, though, the ROCm build code is a huge mess of unnecessary
complexity. We can improve it, but there is no quick fix. To remove
complex code, you need to know that the things you're deleting aren't
fulfilling some requirement on some other platform. Unfortunately, there
is no comprehensive listing of ROCm build requirements or supported
platforms. Understanding why a line of code exists requires detective work.
I spent the past year painstakingly studying the rocSOLVER build code. I
tracked down why each line was added and checked if it was still
relevant. Over the course of 2021, I removed hundreds of lines of
unnecessary CMake. Simplifying the build code is not a fast process. At
least, not in upstream.
In the last jitsi meeting, we were joking about how Securitas could
make a visit to improve Debian workflows: a senior Kitware consultant
could maybe have an useful holistic(tm) CMake overhauling mission at
AMD ROCm team? ROCm is an awesome piece of software, it is sad seing it
being held back by its build system.
Kitware is great. I get the impression it was not an easy process, but
they managed to organize some of the low level details of the ROCm stack
when building the HIP language support for CMake 3.21. We're not using
CMake's HIP language support yet in the math libraries, but I think it
will simplify a lot.
Sincerely,
Cory Bloor
Reply to: