Re: The XDNA NPU stack
Hi Cordell,
Thanks for the summary on AMD's NPUs.
The current situation of NPU on linux looks somewhat messy and not
mature. For GPUs, there are just three major vendors, and their software
stack are relatively more mature than NPUs. In contrast, the number of
NPU vendors is too many, and the support level on linux differs. Even
worse, each vendor seems to keep one copy of their own fork of LLVM that
is unlikely to merge upstream.
I will personally keep an eye on the news, and I do not have plan to
work on NPU packages myself since I see no signal of ecosystem maturity.
Of course, potential contributors are welcome to push things forward.
On 7/30/25 6:53 PM, Cordell Bloor wrote:
Hello,
I know there were folks interested in the XDNA NPUs found in AMD Ryzen
APUs [1], so I've been collecting what information I can about what
the driver, compiler, and runtime stack looks like for that. I've
created a wiki page where anyone interested in working towards AMD NPU
enablement on Debian can fill in more details as they discover them [2].
1. As has been mentioned in the past, the lowest level component is
the XDNA Driver, which was upstreamed into the Linux Kernel in 6.14.
I'm uncertain of whether the firmware has been packaged for Debian and
if the necessary KConfigs have been enabled in the Debian kernel package.
2. There are xrt, xri and vart packages in Debian, which provide the
XDNA Runtime, but Debian currently only builds small pieces of the
runtime and the packages are very out-of-date.
3. The AIE2 backend for LLVM (i.e., the Peano compiler) needs to be
packaged or enabled in the existing LLVM package [3]. This compiler is
used to build code for the NPU processor. It is developed as an LLVM
fork, but it is intended to be upstreamed into LLVM. The upstreaming
was announced a year ago, but I haven't checked on the status of that
effort.
4. The IRON API and MLIR AI Engine provide the interface used by AI
applications and they need to be built and packaged for Debian [4].
5. The XDNA backend for the ONNX Runtime needs to be built and enabled
in Debian.
This information (and more) has been captured in the wiki I linked
[2]. See also Getting Started for AMD Ryzen AI on Linux [5] and the
IRON AIE Programming Guide [6] for more information and a guide to
installing and using the upstream-provided XDNA components via pip
and/or building from source.
Sincerely,
Cory Bloor
[1]: https://lists.debian.org/debian-ai/2025/01/msg00051.html
[2]: https://wiki.debian.org/XDNA
[3]: https://github.com/Xilinx/llvm-aie
[4]: https://github.com/Xilinx/mlir-aie
[5]: https://xilinx.github.io/mlir-aie/
[6]: https://github.com/Xilinx/mlir-aie/tree/main/programming_guide
Reply to: