Re: AI and NPUs? AMD AI Max+ 395? Relation betwene ROCm and NPU?
Hi Mo,
On 2025-01-14 19:54, M. Zhou wrote:
However I'm very unfamiliar to the NPUs from the software side.
https://www.amd.com/en/technologies/xdna.html
Is seems irrelevant to ROCm right?
That is correct. The XDNA NPU is based AMD Xilinx technology and uses
the Xilinx AI runtime software stack rather than the ROCm software
stack. With that said, the AMD AI Max+ (aka Strix Halo) will also have a
relatively powerful integrated GPU that could be used for compute (using
the same software stack as other AMD GPUs).
What is the software stack for NPU support?
The XDNA driver and firmware has been merged to drm-misc-next for
inclusion in Linux 6.14. The xrt [1], xir [2], and vart [2] runtime
libraries appear to have been packaged for Debian in 2022, but do not
appear to have been actively maintained. Those source packages will need
to be updated to the latest upstream version and enhanced to build the
necessary binary components. I'm not expecting those package
enhancements to be easy as the XDNA toolchain is fairly complex and
relatively immature. I'm hoping that it turns out to be easier than it
looks.
In any case, I tried reaching out to Punit and Nobuhiro of the Debian
Xilinx Team, but I have not heard back from them [4]. With that said, I
have slowly learned that the TLD I use for my email does not have a good
reputation and is often caught in spam filters. :(
Cory, do you know something about the NPU support in Debian?
I know a bit, but Mario Limonciello may be a better contact. He is both
a Debian Maintainer and an AMD engineer working on software support for
AMD AI Max+ CPUs. He's been helping with some of the planning for OS
integration of NPU support.
I'll also note that Framework Computers donated a couple of laptops to
support the testing of AI libraries and applications on Debian. They
have Ryzen 7040 CPUs with XDNA 1 NPUs (~10 TOPS) and are not nearly as
powerful as Ryzen AI Max (~50 TOPS), but they should allow us to
exercise the full AMD NPU stack from the driver to the application
level. I received one of those laptops, as I wanted to use it for
enabling gfx1103 support.
I've not been able to contribute much to Debian on this front, as my
spare time has been consumed by family commitments these past couple
months. With that said, I'm sure I can test a few NPU packages as they
become available.
Sincerely,
Cory Bloor
[1]: https://tracker.debian.org/pkg/xrt
[2]: https://tracker.debian.org/pkg/xir
[3]: https://tracker.debian.org/pkg/vart
[4]: https://lists.debian.org/debian-ai/2024/12/msg00028.html
Reply to: