[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bi-weekly update.



HI, I hope you are well. It is too hot in Japan recently.

Anyway, I want to tell you my updates for these 2 weeks.

1.  Refine Rust crates package
I need to fix some errors with autopkgtest. I think all rust crates seem better than before and I hope these can pass to the Debian archive.

2.  update dependencies tree
https://salsa.debian.org/k1000dai/gsoc-status/-/issues/15
I did some research about dependencies. 
And I found that cpu version is almost there. I will build vllm with least dependencies and will add some recommend packages later.

3. add gguf, xgrammar, compressed-tensor
https://salsa.debian.org/python-team/packages/gguf
https://salsa.debian.org/python-team/packages/xgrammar
https://salsa.debian.org/python-team/packages/compressed-tensors

These are necessary for building vllm. xgrammar ( or some other library like lm-format-enforcer, outlines, guidance) is needed for GUIDED DECODING. At least one of them is necessary for the guided decoding so I choose xgrammar as a first one. 
gguf and compressed-tensors is important for reading model files. gguf treats common models and compressed tensors is important for some quantized models.

4. start vllm packaging ! ( cpu ver )
https://salsa.debian.org/deeplearning-team/vllm
Now, I start packaging vllm!
I think cpu and gpu versions are quite different but also have some common parts ( like copyright, or watch files). I will start with cpu version and get a better understanding of vllm. 

My Next Goal is below. 
* Refine the tokenizers and transformers packages. Add more tests and check some dependencies. (Also I will check otheres one by one, but these two is really important)
* Build vllm with cpu. I think I alraedy have every package for vllm cpu version. 
* Start preparation for gpu packages. ray and Xformer not seem a simple library. 

------------------------------------------------------------------------------------------------------
kouhei.sendai@gmail.com

Kohei Sendai

-------------------------------------------------------------------------------------------


Reply to: