Re: ROCm GPU_TARGETS and GPU_ARCHS and some other points

[Christian BAYLE <bayle@debian.org>]

Hello,

on the same subject I noticed that if I don't set any TARGETS or ARCH

-- hip_version_flat=600443484
-- checking which targets are supported
-- Performing Test COMPILER_HAS_TARGET_ID_gfx908
-- Performing Test COMPILER_HAS_TARGET_ID_gfx908 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx942
-- Performing Test COMPILER_HAS_TARGET_ID_gfx942 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx950
-- Performing Test COMPILER_HAS_TARGET_ID_gfx950 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx10_off3_offgeneric
-- Performing Test COMPILER_HAS_TARGET_ID_gfx10_off3_offgeneric - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx11_offgeneric
-- Performing Test COMPILER_HAS_TARGET_ID_gfx11_offgeneric - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx12_offgeneric
-- Performing Test COMPILER_HAS_TARGET_ID_gfx12_offgeneric - Success
-- Building CK for the following targets: 
gfx908;gfx90a;gfx942;gfx950;gfx10-3-generic;gfx11-generic;gfx12-generic


which is quite different from the documentation suggested

GPU_ARCHS="gfx908;gfx90a;gfx942;gfx1030;gfx1100;gfx1101;gfx1102;gfx1200;gfx1201"

This is the result of the cmake code :

#In order to build just the CK library (without tests and examples) for 
all supported GPU targets
#use -D 
GPU_ARCHS="gfx908;gfx90a;gfx942;gfx1030;gfx1100;gfx1101;gfx1102;gfx1200;gfx1201"
#the GPU_TARGETS flag will be reset in this case in order to avoid 
conflicts.
#
#In order to build CK along with all tests and examples it should be OK 
to set GPU_TARGETS to just 1 or 2 similar architectures.
if(NOT ENABLE_ASAN_PACKAGING)
    if(NOT WIN32 AND ${hip_VERSION_FLAT} LESS 600300000)
        # WORKAROUND: compiler does not yet fully support gfx12 
targets, need to fix version above
        set(CK_GPU_TARGETS 
"gfx908;gfx90a;gfx942;gfx1030;gfx1100;gfx1101;gfx1102")
    elseif(NOT WIN32 AND ${hip_VERSION_FLAT} GREATER_EQUAL 600300000 
AND ${hip_VERSION_FLAT} LESS 600400000)
        set(CK_GPU_TARGETS 
"gfx908;gfx90a;gfx942;gfx1030;gfx1100;gfx1101;gfx1102;gfx1200;gfx1201")
    elseif(NOT WIN32 AND ${hip_VERSION_FLAT} GREATER_EQUAL 600400000 
AND ${hip_VERSION_FLAT} LESS 600443483)
        set(CK_GPU_TARGETS 
"gfx908;gfx90a;gfx942;gfx1030;gfx1100;gfx1101;gfx1102;gfx1200;gfx1201;gfx950")
    elseif(NOT WIN32 AND ${hip_VERSION_FLAT} GREATER_EQUAL 600443483)
        set(CK_GPU_TARGETS 
"gfx908;gfx90a;gfx942;gfx950;gfx10-3-generic;gfx11-generic;gfx12-generic")
    endif()
else()
    #build CK only for xnack-supported targets when using ASAN
    set(CK_GPU_TARGETS "gfx908:xnack+;gfx90a:xnack+;gfx942:xnack+")
endif()


this make me think that there will be for us a correlation with hip version.
do you think it should be integrated in rocm-target-arch or that we keep 
a per distro policy ?
I think i will use the generic targets for composable kernel as it 
should reduce the build time

Regards

Christian



Le 25/10/2025 à 08:19, Cordell Bloor a écrit :
> On 2025-10-16 07:10, Christian BAYLE wrote:
> > I'm currently working on composable-kernel [1] package and have some 
> > questions about GPU_TARGETS and GPU_ARCHS that are used to build the 
> > libraries [....]
> > I could do several build on a per arch base, which has the good 
> > property to build tests and examples, but create conflicting per arch 
> > packages
> >
> > On the other hand the build for all arch takes sometime more than 
> > 40Gb/core than will be difficult to run on autobuilders
> >
> > which set to support ? GPU_ARCHS  or GPU_TARGETS ?
> >
> > are there other packages concerned ? and how do you think it would be 
> > best to deal with this ?
>
> I'm afraid I don't have good answers for you. This may be a case where 
> we just try to put something that we think makes sense into the team 
> repo or into experimental, and rework it based on what we discover 
> trying to integrate it into other libraries.
>
> CK is a key library, but I know very little about it aside from the fact 
> that it is not going to be easy to build. I also fear that different CK 
> reverse dependencies may be picky about what version of CK they require. 
> This is just something that we're going to have to learn as we start 
> trying to make use of it.