Bug#1120707: ITP: lemonade -- Local LLM Serving with GPU and NPU acceleration
Package: wnpp
Severity: wishlist
Owner: Mario Limonciello <superm1@debian.org>
X-Debbugs-Cc: debian-devel@lists.debian.org
* Package name : lemonade
Version : 9.0.2
Upstream Contact: Jeremy Fowers <jeremy.fowsers@amd.com>
* URL : https://lemonade-server.ai/
* License : Apache2
Programming Lang: Python
Description : Local LLM Serving with GPU and NPU acceleration
Lemonade helps users run local LLMs with the highest performance by
configuring state-of-the-art inference engines for their NPUs and GPUs.
There is a variety of support with different models and backends advertised
on https://lemonade-server.ai/.
As we gain support for other related packages like transformers, huggingfacehub
and llama.cpp it will act as a layer for users to easily access models.
I plan to maintain it myself initially, but may talk to the Debian
deep learning team about moving it there later.
Reply to: