Re: CMake musings for ROCm 4.5.2 (Was Re: ROCm packaging session memo) (Was Re: ROCm installation)
Hi Cory,
I added Todd Gamblin, from LLNL and creator of Spack [1].
I have had interesting exchanges with him on slack: I hope that with
his deep knowledge of ROCm Spack packages, this will work for everyone.
I have been bugging devs from other big distros as well: Fedora is here,
Arch, Gentoo told me that they would be lending an eye to the thread.
On 2/11/22 06:18, Cordell Bloor wrote:
Today I learned that there is a public pull request on rocPRIM which
shows the proposed changes to support the FHS. This is a
work-in-progress, and I'm hoping we can trim it down. I'd been
reviewing the (internal) rocBLAS version of this PR and trying to
suggest ways to simplify.
This is good news!
But as far as I see it, it does not solve yet many FHS issues.
There are still in rocprim/rocprim/CMakeLists.txt :
```cmake
rocm_install_targets(
TARGETS rocprim rocprim_hip
***************************************
PREFIX rocprim
***************************************
)
```
and
```cmake
# Export targets
rocm_export_targets(
TARGETS roc::rocprim roc::rocprim_hip
***************************************
PREFIX rocprim
***************************************
DEPENDS PACKAGE hip
NAMESPACE roc::
)
```
which, by default, will install rocPRIM in /usr/rocprim, which is the
monolithic way I am trying to avoid. Reminder, by default cmake
GNUInstallDirs places libs+cmake exports in /usr/lib/<triple>,
and headers in /usr/include[/<package>] (which is my goal).
It does remove the operation symlinking directories.
Liam Wrubleski and I will be helping to
clean this up so it can be easily extended to the various math
libraries. (In fact, he's already started on a PR to consolidate the
backwards-compatibility header generation).
About this second PR, more questions come from me than answers:
- what kind of backward compatibility are you aiming for?
- why wrap headers?
- why not stuff a #pragma_once in all headers, à la cgmb/guardonce?
We know that the code is a bit ugly, but the big question I have is:
I find the code often times pretty smart in the details, what I
question are the design decisions leading to the implementations.
What are the install targets for the stack generally?
If this is :
1. spack for HPC - as far as I know all the spack packages work on the
install assumption /opt/rocm?
2. the FHS for linux distros
3. windows
A general design could be a spack-compliant install by default, as is
currently the case, and with a big cmake knob, install the FHS way, in the
waters of ROCM_INSTALL_FHS=TRUE?
Windows does not interfere much with this.
On the other hand, installing by default FHS, and having spack overhaul
a bit
its packages to tell CMAKE_INSTALL_PREFIX=/opt/rocm could work too?
This would need arbitration, if such a change is ever considered.
does the implemented functionality fulfill Debian's needs?
Well not really at the moment, as explained. We can still make it work with
patches, of course.
I have quite a few points about the hipamd package install that I would
enjoy discussing, but I will make a separate mail (you still use cmake
find_package in module mode [2], while you have the config mode ready
for example - the rocPRIM PR extends the module search paths with
hard-coded strings :/).
Best regards, Maxime
[1] https://tgamblin.github.io/pubs/spack-sc15.pdf
[2] https://cmake.org/cmake/help/latest/command/find_package.html#id4
Reply to: