Bug#1095237: ITP: vllm -- A high-throughput and memory-efficient inference and serving engine for LLMs

["M. Zhou" <lumin@debian.org>]

Package: wnpp
Severity: wishlist
Owner: Mo Zhou <lumin@debian.org>
X-Debbugs-Cc: debian-devel@lists.debian.org, debian-ai@lists.debian.org

* Package name    : vllm
  Version         : 0.7.1
  Upstream Contact: 
* URL             : 
* License         : Apache-2.0
  Programming Lang: Python
  Description     : A high-throughput and memory-efficient inference and serving engine for LLMs

I think this is one of the most important applications in the reverse
dependency tree of pytorch package. vllm has a very large tree of dependencies
and many of them are missing. I'm just setting vllm as a long term goal.

Alternatives are ollama and llama.cpp.

Everything, including vllm's necessary dependencies will be maintained by
debian deep learning team.

Thank you for using reportbug