I successfully built the vllm-cuda version and XFormers for the vllm backend!!
Now, I will check the VLLM in the various settings and do more tests with various GPU settings.
Please let me know if you have any concerns or something I need to do in the end of packaging a quite big project.
Also, now I try to update the numpy and tokenizers witty pyo3 0.25.
Thanks for the big help!
Regards.
------------------------------------------------------------------------------------------------------
Kohei Sendai
-------------------------------------------------------------------------------------------