[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: RFC: Centralized ROCm target list + utility



Hi Christian,

Thanks for working on this. It fills a real need, and I appreciate the thought you've put into the tooling.

On 2025-06-23 13:55, Christian Kastner wrote:
1. Master lists
===============

For each distribution, we ship a newline-separated list of build targets in

     /usr/share/pkg-rocm-tools/data/build-targets/<codename>

which look like this:

    # Comments are supported, blank lines are skipped
    gfx900
    gfx906
    ...

<codename> can be a symlink to some other <codename>.

Currently, the trixie list consists of the typical targets, and unstable
and experimental are symlinks to trixie.

I would note the documented behaviour of rocm_enumerate_devices, as that script is queried by:

  1. The HIP Runtime in hip-config.cmake to specify the default value of GPU_TARGETS / AMDGPU_TARGETS when building for a static library (though when building for dynamic libraries, upstream uses amdgpu-arch [1]),
  2. CMake to specify the default value of CMAKE_HIP_ARCHITECTURES [2], and
  3. hipcc to specify the default build target if no architecture is specified [3]

The description of rocm_enumerate_devices is [4]:

Prints the list of available AMD GCN ISA

    The program collects the list in 3 different ways, in the order of
    precendence:

    1. ROCM_TARGET_LST : a user defined environment variable, set to the path and                        filename where to find the "target.lst" file. This can be
                       used in an install environment with sandbox, where
                       execution of "rocminfo" is not possible.
    2. target.lst : user-supplied text file. This is used in a container setting
                  where ROCm stack may usually not available.
    3. HSA topology : gathers the information from the HSA node topology in
                      /sys/class/kfd/kfd/topology/nodes/
    4. lspci : enumerate PCI bus and locate supported devices from a hard-coded
               lookup table.
    5. rocminfo : a tool shipped with this script to enumerate GPU agents
                available on a working ROCm stack.

As such, I would recommend that your proposed master list follow the format of the rocm target list,
so that one can export:

    export ROCM_TARGET_LST=/usr/share/pkg-rocm-tools/data/build-targets/<codename>

It is admittedly not quite as useful as I'd hoped, as it seems that [1] diverged from [2] and [3]. It would nevertheless be a nice thing to make compatible.

Currently, the trixie list consists of the typical targets, and unstable
and experimental are symlinks to trixie.
Please add gfx942 to experimental, given that HIP 6.1.2 is now available on that distribution. Experimental support for gfx942 was added in ROCm 5.7.1, so most of the libraries can probably already compile for gfx942 (and we need to do updates for the rocm-hipamd transition anyway).

$ rocm-arch-support
   -> Prints space-separated list of targets for the current
     distribution: "gfx803 gfx900 gfx906 gfx908 ..."

We should clearly distinguish build targets from supported GPUs. Consider that users may not see gfx1031 in the current values of this list, but it works fine on Debian because we've implemented the gfx1030 fallback. This will continue when we switch the build target to gfx10-3-generic (or spirv).

It can probably be a separate tool, but the average user may also benefit from a simple `is-my-gpu-supported` command.

$ rocm-arch-support --for-build
   -> Prints semicolon-separated list of targets for the current source
      package: "gfx803;gfx900;gfx906;gfx908;..."
   -> use as eg: AMDGPU_TARGETS="$(rocm-arch-support --for-build)"

I'm bikeshedding, but semi-colon separated is specifically a CMake convention. The option should be `--cmake` or something similar. Other build tools may have different conventions.

Sincerely,
Cory Bloor

[1]: https://github.com/ROCm/clr/blob/rocm-6.4.1/hipamd/hip-config-amd.cmake.in#L86 [2]: https://gitlab.kitware.com/cmake/cmake/-/blob/v4.0.3/Modules/CMakeDetermineHIPCompiler.cmake?ref_type=tags#L296-335 [3]: https://salsa.debian.org/rocm-team/rocm-llvm/-/blob/debian/6.1.2+dfsg-1_exp1/amd/hipcc/bin/hipcc.pl?ref_type=tags#L482 [4]: https://salsa.debian.org/rocm-team/rocminfo/-/blob/debian/6.4.1-1_exp1/rocm_agent_enumerator?ref_type=tags#L245-262


Reply to: