[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: CMake musings for ROCm 4.5.2 (Was Re: ROCm packaging session memo)

Thanks for the CMake discussion, Maxime. People sometimes wonder why I spend so much time on the build system when there's far more interesting problems I could be working on. It's messages like yours that help me explain why it's worth spending time on our build.

On 2022-01-28 17:23, Maxime Chambonnet wrote:
On the right, you can see rocBLAS as it is now.
First, in library/CMakeLists.txt, GNUInstallDirs is included and its
others [3]) are referred-to, which seems nice. But in the children
CMakeLists in library/src, which handles the actual install, the
aforementioned GNUInstallDirs are crippled and dropped for a mix of
CMAKE_SOURCE_DIR and hard-coded paths. Installing a package in
/usr/<package> is considered poor practice in Debian-based distribs.

It doesn't work properly because it wasn't designed to do that in ROCm 4.5. It's only just now that support for any layout besides that of /opt/rocm and spack has become a goal. Upstream support probably won't be coming until ROCm 5.2 or later, but we should be able to get it fixed on Debian long before then by backporting patches (once they exist).

I believe I mentioned in the past that you can correct this problem across all math libraries with minor changes to rocm-cmake. I've recently been made a maintainer for rocm-cmake, so I suppose it's now my job to help facilitate that. The primary question is what file layout Debian wants.

I can imagine this nesting under /usr/rocrand + symlinking to /usr
classical sub-dirs, an operation being a remnant of the day that
/opt/rocm was the main assumption?

/usr/rocrand was the result of my quick-and-dirty sample build script. It's not a serious suggestion of anything. It's a prompt for us to discuss what layout Debian would like for the math libraries. For example, is there a preference between /usr/include/rocrand/rocrand.h and /usr/include/rocrand.h?

As I see it today, rocm-cmake is a gross over-complexity. I must be
short-sighted and you might have good reasons, (on top of the CPackDeb
infrastructure which has root in this repo) such as installations on
Windows, or maybe other Linux distros, for its existence?
I can imagine that much cmake could be cleaned if this was the case!

I have not dug deep enough into rocm-cmake to have a personal opinion. It was authored by a skilled developer, so I default to trusting his judgement.

In general, though, the ROCm build code is a huge mess of unnecessary complexity. We can improve it, but there is no quick fix. To remove complex code, you need to know that the things you're deleting aren't fulfilling some requirement on some other platform. Unfortunately, there is no comprehensive listing of ROCm build requirements or supported platforms. Understanding why a line of code exists requires detective work.

I spent the past year painstakingly studying the rocSOLVER build code. I tracked down why each line was added and checked if it was still relevant. Over the course of 2021, I removed hundreds of lines of unnecessary CMake. Simplifying the build code is not a fast process. At least, not in upstream.

In the last jitsi meeting, we were joking about how Securitas could
make a visit to improve Debian workflows: a senior Kitware consultant
could maybe have an useful holistic(tm) CMake overhauling mission at
AMD ROCm team? ROCm is an awesome piece of software, it is sad seing it
being held back by its build system.

Kitware is great. I get the impression it was not an easy process, but they managed to organize some of the low level details of the ROCm stack when building the HIP language support for CMake 3.21. We're not using CMake's HIP language support yet in the math libraries, but I think it will simplify a lot.

Cory Bloor

Reply to: