Re: RFC: Centralized ROCm target list + utility
Hi Christian,
Thanks for working on this. It fills a real need, and I appreciate the
thought you've put into the tooling.
On 2025-06-23 13:55, Christian Kastner wrote:
1. Master lists
===============
For each distribution, we ship a newline-separated list of build targets in
/usr/share/pkg-rocm-tools/data/build-targets/<codename>
which look like this:
# Comments are supported, blank lines are skipped
gfx900
gfx906
...
<codename> can be a symlink to some other <codename>.
Currently, the trixie list consists of the typical targets, and unstable
and experimental are symlinks to trixie.
I would note the documented behaviour of rocm_enumerate_devices, as that
script is queried by:
1. The HIP Runtime in hip-config.cmake to specify the default value
of GPU_TARGETS / AMDGPU_TARGETS when building for a static library
(though when building for dynamic libraries, upstream uses amdgpu-arch [1]),
2. CMake to specify the default value of CMAKE_HIP_ARCHITECTURES [2], and
3. hipcc to specify the default build target if no architecture is
specified [3]
The description of rocm_enumerate_devices is [4]:
Prints the list of available AMD GCN ISA
The program collects the list in 3 different ways, in the order of
precendence:
1. ROCM_TARGET_LST : a user defined environment variable, set to
the path and
filename where to find the "target.lst" file.
This can be
used in an install environment with sandbox, where
execution of "rocminfo" is not possible.
2. target.lst : user-supplied text file. This is used in a
container setting
where ROCm stack may usually not available.
3. HSA topology : gathers the information from the HSA node topology in
/sys/class/kfd/kfd/topology/nodes/
4. lspci : enumerate PCI bus and locate supported devices from a
hard-coded
lookup table.
5. rocminfo : a tool shipped with this script to enumerate GPU agents
available on a working ROCm stack.
As such, I would recommend that your proposed master list follow the
format of the rocm target list,
so that one can export:
export
ROCM_TARGET_LST=/usr/share/pkg-rocm-tools/data/build-targets/<codename>
It is admittedly not quite as useful as I'd hoped, as it seems that [1]
diverged from [2] and [3]. It would nevertheless be a nice thing to make
compatible.
Currently, the trixie list consists of the typical targets, and unstable
and experimental are symlinks to trixie.
Please add gfx942 to experimental, given that HIP 6.1.2 is now available
on that distribution. Experimental support for gfx942 was added in ROCm
5.7.1, so most of the libraries can probably already compile for gfx942
(and we need to do updates for the rocm-hipamd transition anyway).
$ rocm-arch-support
-> Prints space-separated list of targets for the current
distribution: "gfx803 gfx900 gfx906 gfx908 ..."
We should clearly distinguish build targets from supported GPUs.
Consider that users may not see gfx1031 in the current values of this
list, but it works fine on Debian because we've implemented the gfx1030
fallback. This will continue when we switch the build target to
gfx10-3-generic (or spirv).
It can probably be a separate tool, but the average user may also
benefit from a simple `is-my-gpu-supported` command.
$ rocm-arch-support --for-build
-> Prints semicolon-separated list of targets for the current source
package: "gfx803;gfx900;gfx906;gfx908;..."
-> use as eg: AMDGPU_TARGETS="$(rocm-arch-support --for-build)"
I'm bikeshedding, but semi-colon separated is specifically a CMake
convention. The option should be `--cmake` or something similar. Other
build tools may have different conventions.
Sincerely,
Cory Bloor
[1]:
https://github.com/ROCm/clr/blob/rocm-6.4.1/hipamd/hip-config-amd.cmake.in#L86
[2]:
https://gitlab.kitware.com/cmake/cmake/-/blob/v4.0.3/Modules/CMakeDetermineHIPCompiler.cmake?ref_type=tags#L296-335
[3]:
https://salsa.debian.org/rocm-team/rocm-llvm/-/blob/debian/6.1.2+dfsg-1_exp1/amd/hipcc/bin/hipcc.pl?ref_type=tags#L482
[4]:
https://salsa.debian.org/rocm-team/rocminfo/-/blob/debian/6.4.1-1_exp1/rocm_agent_enumerator?ref_type=tags#L245-262
Reply to: