Bug#1095237: ITP: vllm -- A high-throughput and memory-efficient inference and serving engine for LLMs
Package: wnpp
Severity: wishlist
Owner: Mo Zhou <lumin@debian.org>
X-Debbugs-Cc: debian-devel@lists.debian.org, debian-ai@lists.debian.org
* Package name : vllm
Version : 0.7.1
Upstream Contact:
* URL :
* License : Apache-2.0
Programming Lang: Python
Description : A high-throughput and memory-efficient inference and serving engine for LLMs
I think this is one of the most important applications in the reverse
dependency tree of pytorch package. vllm has a very large tree of dependencies
and many of them are missing. I'm just setting vllm as a long term goal.
Alternatives are ollama and llama.cpp.
Everything, including vllm's necessary dependencies will be maintained by
debian deep learning team.
Thank you for using reportbug
Reply to: