Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++

[Christian Kastner <ckk@debian.org>]

Hi Petter,

On 2024-02-13 08:36, Petter Reinholdtsen wrote:
> I tried building the CPU edition on one machine and run it on another,
> and experienced illegal instruction exceptions.  I suspect this mean one
> need to be careful when selecting build profile to ensure it work on all
> supported Debian platforms.

yeah, that was my conclusion from my first experiments as well.

This is a problem though, since one key point of llama.cpp is to make
best use of the current hardware. If we'd target some 15-year-old amd64
lowest common denominator, we'd go against that.

In my first experiments, I've also had problems with ROCm builds on
hosts without a GPU.

I have yet to investigate if/how capabilities can be generally enabled,
and use determined at runtime.

Another issue that stable is clearly the wrong distribution for this.
This is a project that is continuously gaining new features, so we'd
need to stable-updates.

> I would be happy to help getting this up and running.  Please let me
> know when you have published a git repo with the packaging rules.

I'll push a first draft soon, though it will definitely not be
upload-ready for the above reasons.

Best,
Christian