I also added WORKLOG.md, where I will write some notes and what I did.
Also, I will send my progress bi-weekly to you and discuss next steps.
Now, I want to decide the vllm version to package during this GSoC project.
I think there are two options.
First is to follow the latest and catch up on every dependency for that.
Second, and I think this might be much easier, is to follow the v0.8.4.
v0.8.5 (latest version) requires tokenizers >= 0.21.1 and tokenizers v0.21.1 requires pyo3 24.0 and numpy-rust 0.24.0; however, the rust team says that backporting to pyo3 0.22 (currently in Debian unstable) or upgrading pyo3 to 0.24 after the current Debian release is done in a few months is required.
In contrast, vllm v0.8.4 requires tokenizers >= 0.19.1 and tokenizers v0.20.3 requires pyo3 22.0 and numpy-rst 0.22.0 which seems to be good for avoiding repackaging pyo3 crate.
I also think it’s important to keep up with the latest developments in LLMs since the field is evolving so quickly. How do you think about this?
Best regards,
Kohei Sendai
------------------------------------------------------------------------------------------------------
University of Tokyo, Faculty of Engineering, system innovation.
------------------------------------------------------------------------------------------