Re: Advice on ARMv9 resources?
On Fri, May 23, 2025 at 08:56:11PM +0200, Christian Kastner wrote:
> I maintain llama.cpp and its underlying tensor library ggml. ggml has
> various CPU and GPU backends that are heavily optimized, and include
> various forms of dynamic dispatching. My goal for configuring the build
> is to have something that maximally exploits CPU features on recent
> hardware, whilst supporting everything down to the baseline.
>
> I started with amd64 and found a simple and effective solution: I have a
> very recent CPU supporting up to AVX512, and using QEMU+kvm, I can
> efficiently emulate older CPUs in a VM, and run tests there.
>
> Now I'd like to do the same with ARM, but I wonder how to best approach
> this.
>
> The analogous solution would be to either get access to, or purchase a
> system with an ARMv9 CPU, and use that to emulate older CPUs. However I
> don't know how to best go about that.
>
> I see that Graviton4 is ARMv9, so that would be interesting, but I don't
> know if they'll support nested kvm. I also looked into SBCs but the only
> ARMv9 I found was the Radxa Orion O6, which was announced only months
> ago, so I'm not sure that it would be well supported by Debian yet.
> Though they seem to want to support EFI which in my eyes is a huge plus.
>
> I would appreciate any ideas or suggestions that might help me achieve
> my goal. Most importantly, am I perhaps being too aggressive in looking
> for ARMv9? Should I settle for something less recent?
Which older ARM chips are you hoping to emulate?
If you mean doing what qemu+kvm can do on x64 where you can hide CPU
feature flags to pretend to have an older CPU, I am not so sure you
can do that with ARM. It does appear when searching that most 64bit
arm chips do support 32 bit still, so perhaps it would work. I know at
least a few of the high end server chips dropped 32 bit support though,
but apparently it is not a common design choice yet.
That Orion O6 does look pretty neat. It seems they even claim it comes
with Debian although with a custom kernel installed and probably a few
other extra packages added.
--
Len Sorensen
Reply to: