HI, I hope you are well. It is too hot in Japan recently.
Anyway, I want to tell you my updates for these 2 weeks.
1. Refine Rust crates package
I need to fix some errors with autopkgtest. I think all rust crates seem better than before and I hope these can pass to the Debian archive.
2. update dependencies tree
I did some research about dependencies.
And I found that cpu version is almost there. I will build vllm with least dependencies and will add some recommend packages later.
3. add gguf, xgrammar, compressed-tensor
These are necessary for building vllm. xgrammar ( or some other library like lm-format-enforcer, outlines, guidance) is needed for GUIDED DECODING. At least one of them is necessary for the guided decoding so I choose xgrammar as a first one.
gguf and compressed-tensors is important for reading model files. gguf treats common models and compressed tensors is important for some quantized models.
4. start vllm packaging ! ( cpu ver )
Now, I start packaging vllm!
I think cpu and gpu versions are quite different but also have some common parts ( like copyright, or watch files). I will start with cpu version and get a better understanding of vllm.
My Next Goal is below.
* Refine the tokenizers and transformers packages. Add more tests and check some dependencies. (Also I will check otheres one by one, but these two is really important)
* Build vllm with cpu. I think I alraedy have every package for vllm cpu version.
* Start preparation for gpu packages. ray and Xformer not seem a simple library.
------------------------------------------------------------------------------------------------------
Kohei Sendai
-------------------------------------------------------------------------------------------