[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Building llama.cpp for AMD GPU using only Debian packages?



Hi Petter,

On 2024-07-01 03:12, Petter Reinholdtsen wrote:
Anyone here know which set of packages to install from Debian, either
stable, experimental or unstable, to build llama.cpp and whisper.cpp for
use with a AMD GPU?  I fail to find the package names listed in various
recipes, and suspect different package names are needed.

I wrote a guide a while ago [1], but it is out-of-date. The old instructions needed to be updated because Debian is that now using clang-17 rather than clang-15. The rest of the change to the build instructions were because llama-cpp has adjusted their build process (and I must say that I like their changes). Here is a new set of instructions for building and running a recent llama-cpp version on Debian Unstable:

apt -y update
apt -y upgrade
apt -y install git wget hipcc libhipblas-dev librocblas-dev cmake build-essential
wget https://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF/resolve/main/dolphin-2.2.1-mistral-7b.Q5_K_M.gguf?download=true -O dolphin-2.2.1-mistral-7b.Q5_K_M.gguf
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
git checkout b3267
HIPCXX=clang-17 cmake -H. -Bbuild -DGGML_HIPBLAS=ON -DCMAKE_HIP_ARCHITECTURES="gfx803;gfx900;gfx906;gfx908;gfx90a;gfx942;gfx1010;gfx1030;gfx1100;gfx1101;gfx1102" -DCMAKE_BUILD_TYPE=Release
make -j32 -C build
build/bin/llama-cli -ngl 32 --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -m ../dolphin-2.2.1-mistral-7b.Q5_K_M.gguf --prompt "Once upon a time"

I'll take a look at whisper-cpp too, but I'm out of time for the moment.

Sincerely,
Cory Bloor

[1]: https://lists.debian.org/debian-ai/2024/02/msg00067.html


Reply to: