Bug#1065572: ITP: llm -- Access large language models from the command-line
On 2024-03-06 16:11:55, Antoine Beaupré wrote:
> On 2024-03-06 22:06:18, Petter Reinholdtsen wrote:
>> [Antoine Beaupre]
>>> A CLI utility and Python library for interacting with Large Language
>>> Models, both via remote APIs and models that can be installed and run
>>> on your own machine.
>>
>> Is the option to run models locally related to llama.cpp, ref
>> <URL: https://bugs.debian.org/1063673 >?
>
> I am not absolutely sure, but yes, I would assume that's one of the
> backends llm can use.
For what it's worth, I have just checked again, and it's actually the
first plugin in the "plugin directory" here:
https://llm.datasette.io/en/stable/plugins/directory.html#plugin-directory
To get *that* plugin working, however, we'd also need to package a shim
library as well:
https://github.com/abetlen/llama-cpp-python
There's a whole forest of plugins with their own dependency chains in
there, I'd love to get some help in packaging those.
--
We all pay for life with death, so everything in between should be
free.
- Bill Hicks
Reply to: