Re: bi weekly update
Hi Kohei,
On 7/13/25 10:15 AM, 千代航平 wrote:
Sorry for a bit delay.
Now, the big progress is I could build vllm with cpu version and it works!
I'm really happy about this.
Wonderful! I'll continue giving you feedback on pending packages and
provide feedbacks to you.
We can upload the CPU version first and pass the NEW queue. That means
the vllm-cpu and vllm-cuda can be organized like how src:pytorch and
src:pytorch-cuda does.
Of course, my next goal is gpu version.
However, I struggle with ray and xformers.
Is ray a mandatory dependency?
First, ray is build with bazel. However, it has strict version
dependencies with bazel it self, and it download source code from the
Internet. Copy the code by hand might not be a big issue, but the
Bazel version is critical for me. I'm struggling with this problem
almost all week....
Bazel's design principle does not fit well in the Debian's context. It
is more suitable for source-based distributions or distributions that do
not worry much about unix file system hierarchy, etc. Updating bazel
alone can be already a new gsoc project. If possible, we'd better avoid
the packages depending on bazel, and avoid touching bazel at all, for
the current project.
Also, xformers depend on flash-attention.
https://github.com/Dao-AILab/flash-attention/tree/3ba6f826b199ff68aa9e9139a46280160defa5cd.
I think I need to build flash-attn first, but is it right? I
have experience using flash-attn and I know it depends on torch and
cuda versions.
Dependency on torch-cuda should not be a problem. What you need should
be just
libtorch-cuda-dev, python3-torch-cuda, nvidia-cuda-toolkit-gcc
Do you mean flash-attn has any particular version requirement on torch/cuda?
Thanks for your help, I could upload some packages and will upload more.
we discussed the issues
https://salsa.debian.org/k1000dai/gsoc-status/-/issues and next
package might be ndarray. ndarray depends on
https://packages.debian.org/search?keywords=librust-portable-atomic-util-dev
and rust team upload this and accepted to experimental. I think it is
ready to upload.
Will continue processing the queue when I'm available.
Also, I will send the MR in llama.cpp for packaging gguf-py.
Great! Christian can help merging additional modifications if you have any.
I know you are busy, but I hope I can get some advice on handling ray
and xformers.
As suggested above. Don't worry, bazel is a hard bit for everyone. You
are making a good progress.
Please keep up the good work!
Regards.
-----------------------------------------------------------------------------------------------------
kouhei.sendai@gmail.com
Kohei Sendai
-------------------------------------------------------------------------------------------
Reply to: