I found the HIP library itself to be a bit tricky. There are problems within clang if we put the
HIP headers in /usr/include. I had a discussion with a couple of LLVM contributors about that in a Compiler Explorer issue.
This is indeed the crux of the problem, the lower part is pretty
fine and mostly done...
- although from a debian-packaging point of view it is far from
refined enough -
... and I'd expect the higher libraries to go smoothly.
I made little progress on hip building, and did not reach the
points you mentioned above yet.
Here is some feedback in a mardown fashion,
hopefully it can be some food for thought before meeting.
If such gloubi-boulga is not welcome, don't hesitate to tell :) !
Also, attached is some spreadsheet data which is helpful at my
place.
BR, Maxime
---------------------------------------------------------------------------------------
# 1. wonders for AMD
# 2. wonders for Debian DM-DDs
---------------------------------------------------------------------------------------
# 1. wonders for AMD
Generally way too much cmake.
Is it welcome to patch out the CPackDeb infrastructure so as to
get
in the Debian way? CPackDeb feels very limited compared to the
specialized
packaging tooling that Debian have.
## general
What decides what is in the ROCm manifest (default.xml in ROCm
repo)
and what is not?
Generally not FHS-compliant to install in /opt/rocm
Will it be possible to compile all without llvm nor clang forks
soon?
## comgr (repo compiler-support)
Comgr : build errors that seem to have been patched, but not yet
released. not
all #include "llvm/MC/TargetRegistry.h" does not compile against
llvm-13 testing
Ugly hack works
## hip
Dependencies:
- rocm-opencl-runtime
- rocclr
- hipcc
- ... (not figured out yet)
Why do you need rocm-opencl to build hip?
Too much cmake:
```
maxzor@maxdeb:~/rocm/try_raw_hip_build$ ls
hip hipamd ROCclr ROCm-OpenCL-Runtime
maxzor@maxdeb:~/rocm/try_raw_hip_build$ cloc .
---------------------------------------------------------------------------------------
Language files blank
comment code
---------------------------------------------------------------------------------------
C++ 808 34860
39777 185352
C/C++ Header 465 25517
30066 140971
CMake 1007 11651
29331 79892
```
### hipcc
hip currently uses a vendored hipcc binary in hip/bin.
Is building hipcc (repo not in the manifest) required by Debian?
It compiles fine AFAIK.
### rocm-opencl (including icd) + rocclr
There is a build-dependency circular reference between
rocm-opencl
(-runtime, not the deprecated -compiler) and rocm-clr.
CMake generally broken here. Hacked in respective CMakeLists.txt :
rocclr:
```
include_directories("/home/maxzor/rocm/amd_sources/radeonopencompute/ROCm-OpenCL-Runtime/khronos/headers/opencl2.2"
)
include_directories("/home/maxzor/rocm/amd_sources/radeonopencompute/ROCm-OpenCL-Runtime/khronos/headers/"
)
include_directories("/home/maxzor/rocm/amd_sources/radeonopencompute/ROCm-OpenCL-Runtime"
)
```
=> Produces a librocclr.a that you copy to your /usr/lib
rocm-opencl-runtime:
```
include_directories("/home/maxzor/rocm/amd_sources/radeonopencompute/ROCm-OpenCL-Runtime/khronos/headers/opencl2.2")
include_directories("/home/maxzor/rocm/amd_sources/radeonopencompute/ROCm-OpenCL-Runtime/khronos")
include_directories("/home/maxzor/rocm/amd_sources/radeonopencompute/ROCm-OpenCL-Runtime/khronos/headers")
include_directories("/home/maxzor/rocm/amd_sources/radeonopencompute/ROCm-OpenCL-Runtime")
include_directories("/home/maxzor/rocm/amd_sources/rocm-developer-tools/ROCclr/include/")
include_directories("/home/maxzor/rocm/amd_sources/rocm-developer-tools/ROCclr")
include_directories("/home/maxzor/rocm/amd_sources/rocm-developer-tools/ROCclr/elf")
include_directories("/home/maxzor/rocm/amd_sources/rocm-developer-tools/ROCclr/device/gpu/gslbe/src/rt/")
```
=> Produces target common
./khronos/icd/libOpenCL.so
./tools/cltrace/libcltrace.so
./tools/clinfo/clinfo
./amdocl/libamdocl64.so
## ROCk
Is amdgpu <==> ROCk?
(Is KFD fully in AMDGPU now?)
My compiled rocminfo says:
```
maxzor@maxdeb:~$ sudo rocminfo
ROCk module is loaded
KFD does not support xnack mode query.
ROCr must assume xnack is disabled.
HSA Error: Incompatible kernel and userspace, AMD Radeon VII
disabled. Upgrade amdgpu.
terminate called after throwing an instance of
'std::bad_function_call'
what(): bad_function_call
Aborted
```
# 2. wonders for Debian DM-DDs
Build thunk static or dynamic? I went for shlib.
amd_device_libs bitcode = proprietary blobs?
How is it different from the firmware .bin microcode? no...
compiled shaders?
In libhsa-runtime debs, why no libhsa-runtime64-config.cmake to
/usr/lib/x86_64-linux-gnu/cmake/hsa-runtime64 ?
Thinking about onomastics. Better HSA_RUNTIME than HSA-RUNTIME !(?
I think not)
no runtime"64" - replace with arch ?
Have the debian package names be the
1.cmake targets 2.repo names 3 debianized names?
Install dirs?
/usr/lib/x86_64-linux-gnu/libhsakmt.so
/usr/lib/libhsa-runtime64.so
Libhsa runtime dev why checksums on all but .so's?
## Project management
### Current tools
Repo: salsa
# TODO
Blob list
Task board: Trello? https://trello.com/b/NX4ERZEE/main
# Ideas
- SQLite database cloc urls... cf .csv?
- Integration with draw.io dep graph (apt-rdepends + vcg not
super sexy)?
- Git repo for ROCm is great!
Salsify [git-repo](https://gerrit.googlesource.com/git-repo/),
or the contrary?
- Work on live branches rather than snapshots? Dunno how with
gbp.
- CI/CD?
https://about.gitlab.com/blog/2016/10/12/automated-debian-package-build-with-gitlab-ci/
in manifest,region,amd project,amd github about,debian package name,package status,comment,sloc,upstream url,salsa url,remote,remote url x,rocm,rock-kernel-driver,,,no package plan,,,https://github.com/radeonopencompute/rock-kernel-driver,,roc-github,https://github.com/RadeonOpenCompute/ x,rocm,roct-thunk-interface,,,experimental,,,https://github.com/radeonopencompute/roct-thunk-interface,,roc-github,https://github.com/RadeonOpenCompute/ x,rocm,rocr-runtime,,,experimental,,,https://github.com/radeonopencompute/rocr-runtime,,roc-github,https://github.com/RadeonOpenCompute/ x,rocm,rocm_smi_lib,,,experimental,,,https://github.com/radeonopencompute/rocm_smi_lib,,roc-github,https://github.com/RadeonOpenCompute/ x,rocm,rocm-cmake,,,experimental,,,https://github.com/radeonopencompute/rocm-cmake,,roc-github,https://github.com/RadeonOpenCompute/ x,rocm,rocminfo,,,experimental,,,https://github.com/radeonopencompute/rocminfo,,roc-github,https://github.com/RadeonOpenCompute/ x,rocm,rocprofiler,,,,,,https://github.com/rocm-developer-tools/rocprofiler,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,rocm,roctracer,,,,,,https://github.com/rocm-developer-tools/roctracer,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,rocm,rocm-opencl-runtime,,,,,,https://github.com/radeonopencompute/rocm-opencl-runtime,,roc-github,https://github.com/RadeonOpenCompute/ x,rocm,opencl-icd-loader,,,,,,https://github.com/khronosgroup/opencl-icd-loader,,khronosgroup,https://github.com/KhronosGroup/ x,rocm,clang-ocl,,,,,,https://github.com/radeonopencompute/clang-ocl,,roc-github,https://github.com/RadeonOpenCompute/ x,hip,hip,,,,,,https://github.com/rocm-developer-tools/hip,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,hip,hipamd,,,,,,https://github.com/rocm-developer-tools/hipamd,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,hip,hip-examples,,,,,,https://github.com/rocm-developer-tools/hip-examples,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,hip,rocclr,,,,,,https://github.com/rocm-developer-tools/rocclr,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,hip,hipify,,,,,,https://github.com/rocm-developer-tools/hipify,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,llvm,llvm-project,,,no package plan,,,https://github.com/radeonopencompute/llvm-project,,roc-github,https://github.com/RadeonOpenCompute/ x,llvm,rocm-device-libs,,,experimental,,,https://github.com/radeonopencompute/rocm-device-libs,,roc-github,https://github.com/RadeonOpenCompute/ x,llvm,atmi,,,,,,https://github.com/radeonopencompute/atmi,,roc-github,https://github.com/RadeonOpenCompute/ x,llvm,rocm-compilersupport,,,experimental,,,https://github.com/radeonopencompute/rocm-compilersupport,,roc-github,https://github.com/RadeonOpenCompute/ x,llvm,rocr_debug_agent,,,,,,https://github.com/rocm-developer-tools/rocr_debug_agent,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,llvm,rocm_bandwidth_test,,,,,,https://github.com/radeonopencompute/rocm_bandwidth_test,,roc-github,https://github.com/RadeonOpenCompute/ x,llvm,half,,,,,,https://github.com/rocmsoftwareplatform/half,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,llvm,rcp,,,,,,https://github.com/gpuopen-tools/radeon_compute_profiler,,gpuopen-tools,https://github.com/GPUOpen-Tools/ x,gdb,rocgdb,,,,,,https://github.com/rocm-developer-tools/rocgdb,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,gdb,rocdbgapi,,,,,,https://github.com/rocm-developer-tools/rocdbgapi,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,libraries,rocblas,,,,,,https://github.com/rocmsoftwareplatform/rocblas,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,tensile,,,,,,https://github.com/rocmsoftwareplatform/tensile,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,hipblas,,,,,,https://github.com/rocmsoftwareplatform/hipblas,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,rocfft,,,,,,https://github.com/rocmsoftwareplatform/rocfft,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,hipfft,,,,,,https://github.com/rocmsoftwareplatform/hipfft,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,rocrand,,,,,,https://github.com/rocmsoftwareplatform/rocrand,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,rocsparse,,,,,,https://github.com/rocmsoftwareplatform/rocsparse,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,rocsolver,,,,,,https://github.com/rocmsoftwareplatform/rocsolver,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,hipsolver,,,,,,https://github.com/rocmsoftwareplatform/hipsolver,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,hipsparse,,,,,,https://github.com/rocmsoftwareplatform/hipsparse,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,rocalution,,,,,,https://github.com/rocmsoftwareplatform/rocalution,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,miopengemm,,,,,,https://github.com/rocmsoftwareplatform/miopengemm,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,miopen,,,,,,https://github.com/rocmsoftwareplatform/miopen,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,miopentensile,,,,,,https://github.com/rocmsoftwareplatform/miopentensile,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,rccl,,,,,,https://github.com/rocmsoftwareplatform/rccl,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,mivisionx,,,,,,https://github.com/gpuopen-professionalcompute-libraries/mivisionx,,gpuopen-libs,https://github.com/GPUOpen-ProfessionalCompute-Libraries/ x,libraries,rocthrust,,,,,,https://github.com/rocmsoftwareplatform/rocthrust,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,hipcub,,,,,,https://github.com/rocmsoftwareplatform/hipcub,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,rocprim,,,,,,https://github.com/rocmsoftwareplatform/rocprim,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,hipfort,,,,,,https://github.com/rocmsoftwareplatform/hipfort,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,amdmigraphx,,,,,,https://github.com/rocmsoftwareplatform/amdmigraphx,,rocm-swplat,https://github.com/ROCmSoftwarePlatform/ x,libraries,rocmvalidationsuite,,,,,,https://github.com/rocm-developer-tools/rocmvalidationsuite,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,aomp,aomp,,,,,,https://github.com/rocm-developer-tools/aomp,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,aomp,aomp-extras,,,,,,https://github.com/rocm-developer-tools/aomp-extras,,rocm-devtools,https://github.com/ROCm-Developer-Tools/ x,aomp,flang,,,,,,https://github.com/rocm-developer-tools/flang,,rocm-devtools,https://github.com/ROCm-Developer-Tools/