Re: Inquiry about GSoC 2025 Debian ROCm proposal

Hi Cordell,

Thanks for your response. Although I don't have the appropriate hardware, I believe it's still possible for me to write the necessary files (e.g., debian/control) to set up the package. Someone with access to the required GPUs could then use my files to build and test whether everything works correctly before merging.

I'm not sure if this approach is viable—please let me know if there's any issue with my reasoning.

Originally, I intended to work on onnxruntime and its dependency for GSoC proposal since I have some experience with ONNX [1], but both onnx and onnxruntime appear to be up to date in this page [2]. Based on your suggestion, would it make sense for me to switch to packaging llama.cpp and its dependencies for Debian?

[1]: https://github.com/ExplorerRay/onnxinfo

[2]: https://qa.debian.org/developer.php?email=debian-ai%40lists.debian.org

On Wed, Mar 26, 2025 at 3:55 PM Cordell Bloor <cgmb@slerp.xyz> wrote:

Hi Bo-Jun and Xuanteng,

On 2025-03-26 01:13, Xuanteng Huang wrote:
> try to package a ROCm component not yet included in Debian.

If you have access to appropriate hardware, I might suggest packaging
rocWMMA [1]. It is a library for accelerating mixed-precision matrix
multiply-accumulate operations. It is specifically designed to help take
advantage of Wave Matrix Multiply Accumulate (WMMA) instructions in RDNA
3/4 GPUs and Matrix Fused-Multiply-Add (MFMA) instructions in CDNA 1/2/3
GPUs.

I reviewed the library a couple years ago but I didn't get any further
than creating an empty Salsa repo [2] because there weren't any
libraries or applications that used rocWMMA. That seems to have changed,
as llama.cpp will now look for it when GGML_HIP_ROCWMMA_FATTN is ON [3].

Sincerely,
Cory Bloor

[1]: https://github.com/ROCm/rocWMMA
[2]: https://salsa.debian.org/rocm-team/rocwmma
[3]:
https://github.com/ggml-org/llama.cpp/blob/b4958/ggml/src/ggml-hip/CMakeLists.txt#L42C5-L42C27

Best regards,
黃柏竣 / Bo-Jun Huang

國立陽明交通大學資訊工程學系 (NYCU CS)

Department of Computer Science,

National Yang Ming Chiao Tung University