Hi Petter,
Anyone here know which set of packages to install from Debian, either stable, experimental or unstable, to build llama.cpp and whisper.cpp for use with a AMD GPU? I fail to find the package names listed in various recipes, and suspect different package names are needed.
I wrote a guide a while ago [1], but it is out-of-date. The old instructions needed to be updated because Debian is that now using clang-17 rather than clang-15. The rest of the change to the build instructions were because llama-cpp has adjusted their build process (and I must say that I like their changes). Here is a new set of instructions for building and running a recent llama-cpp version on Debian Unstable:
apt -y update apt -y upgrade apt -y install git wget hipcc libhipblas-dev librocblas-dev cmake build-essential wget https://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF/resolve/main/dolphin-2.2.1-mistral-7b.Q5_K_M.gguf?download=true -O dolphin-2.2.1-mistral-7b.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp git checkout b3267 HIPCXX=clang-17 cmake -H. -Bbuild -DGGML_HIPBLAS=ON -DCMAKE_HIP_ARCHITECTURES="gfx803;gfx900;gfx906;gfx908;gfx90a;gfx942;gfx1010;gfx1030;gfx1100;gfx1101;gfx1102" -DCMAKE_BUILD_TYPE=Release make -j32 -C build build/bin/llama-cli -ngl 32 --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -m ../dolphin-2.2.1-mistral-7b.Q5_K_M.gguf --prompt "Once upon a time"
I'll take a look at whisper-cpp too, but I'm out of time for the
moment.
Sincerely,
Cory Bloor
[1]: https://lists.debian.org/debian-ai/2024/02/msg00067.html