Re: Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++
- To: Christian Kastner <ckk@debian.org>, debian-ai@lists.debian.org
- Subject: Re: Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and others) in pure C/C++
- From: "M. Zhou" <lumin@debian.org>
- Date: Sat, 01 Feb 2025 18:37:27 -0500
- Message-id: <[🔎] 2f158aa02fac5d00dcdcfc8a6ce0ee2a147bc3c0.camel@debian.org>
- In-reply-to: <[🔎] c9735f7c-982a-4e81-a048-bc588833dccf@debian.org>
- References: <d373f55c-2869-490b-aeaf-0fba8c10c02e@debian.org> <d373f55c-2869-490b-aeaf-0fba8c10c02e@debian.org> <sa6mss4bytd.fsf@hjemme.reinholdtsen.name> <fdedee66-9a55-475e-9e23-acfdfc351025@debian.org> <d373f55c-2869-490b-aeaf-0fba8c10c02e@debian.org> <sa65xxw4jhn.fsf@hjemme.reinholdtsen.name> <22d3e2d2-cfbd-431d-9211-e902ac3dfe4b@debian.org> <22d3e2d2-cfbd-431d-9211-e902ac3dfe4b@debian.org> <d373f55c-2869-490b-aeaf-0fba8c10c02e@debian.org> <de29a469-6c9b-4025-bbed-988e10dc5a38@slerp.xyz> <0aa4f182-da25-4ba5-8d9f-a1d1f8ad9221@debian.org> <ece647c1-3dba-4737-a215-c93112990fe4@debian.org> <7976e018-a547-4bba-82ba-13847980356e@debian.org> <efae84f0-dfa9-4cd2-a869-752ae1bd22cd@debian.org> <[🔎] c9735f7c-982a-4e81-a048-bc588833dccf@debian.org>
On Sat, 2025-02-01 at 19:34 +0100, Christian Kastner wrote:
>
> Using this new approach, I was elated to see that it doesn't just work
> perfectly, it also works with non-standard library paths (lacking
> stability, llama.cpp's libraries are kept private, for now). And the
> baseline build should work fine for any other dynamic linker
> implementation, so there is really no meaningful downside to using
> hwcaps.
Thanks for looking into it. The result looks not only effective but
actually really perfect, and is the first example (as far as I know)
where hwcap is very suitable for Debian package. Good job! My concern
on Debian ISA and performance is gone.
Reply to: