Re: QA testing for llama.cpp
Hi Cory, Petter,
On 2025-05-08 22:05, Cordell Bloor wrote:
> AMD is looking into doing QA for llama.cpp on Fedora. I'd love to tell
> them that they can also start on QA for llama.cpp on Debian and Ubuntu
> at the same time. The results of this QA testing would be used by AMD to
> file internal bugs on the upstream projects (and their ROCm
> dependencies).
that's great to hear!
As llama.cpp interactions are via CLI or API, I assume this QA would be
automated to some extent, or even entirely?
If so, I'd love to talk to them on this, or have patches/MRs submitted,
and see what can be integrated in our package as autopkgtests, so that
our own CI and users can run this as well.
I know this would be a longer process, as it's all but certain that DFSG
non-free data is involved. But while we wait for the opportunity of a
good free data set, we might as well work on the process so that when we
have one, everything else is already in place.
I'm already working towards this process llama.cpp's upstream data sets
(though I've been running into ENOTIME for the past two weeks [1]).
> I can ask AMD QA to file public bugs on bugs.debian.org
> and launchpad.net as well, if desired.
That would be helpful for users, though it would be great if those bugs
would include 'forwarded' tags to the relevant upstream trackers you
mention above.
> Are there any barriers preventing an upload to unstable? If so, is there
> anything I can help with?
The stuff I'm working on needs to go through experimental because of
NEW, but none of it blocks an upload to unstable, so I stashed my other
changes, updated to newest releases, and uploaded.
On 2025-05-08 23:09, Petter Reinholdtsen wrote:
> I am not aware of any. As neither ggml nor llama.cpp are headed for
> Trixie (deadline passed), they should not affect testing migration in
> any way.
ggml's acceptance didn't make the cut for trixie, so I didn't see the
above as too urgent.
In any case, my goal was to provide frequent backports of ggml and
llama.cpp anyway, as these are such fast-moving targets.
Best,
Christian
[1]: Sadly, I didn't even have time to process, let alone contribute to
the GR thread on -vote :( Thought it's one of the most important
discussions we can have in Debian right now.
Reply to: