Advice on ARMv9 resources?

[Christian Kastner <ckk@debian.org>]

Hi,

I maintain llama.cpp and its underlying tensor library ggml. ggml has
various CPU and GPU backends that are heavily optimized, and include
various forms of dynamic dispatching. My goal for configuring the build
is to have something that maximally exploits CPU features on recent
hardware, whilst supporting everything down to the baseline.

I started with amd64 and found a simple and effective solution: I have a
very recent CPU supporting up to AVX512, and using QEMU+kvm, I can
efficiently emulate older CPUs in a VM, and run tests there.

Now I'd like to do the same with ARM, but I wonder how to best approach
this.

The analogous solution would be to either get access to, or purchase a
system with an ARMv9 CPU, and use that to emulate older CPUs. However I
don't know how to best go about that.

I see that Graviton4 is ARMv9, so that would be interesting, but I don't
know if they'll support nested kvm. I also looked into SBCs but the only
ARMv9 I found was the Radxa Orion O6, which was announced only months
ago, so I'm not sure that it would be well supported by Debian yet.
Though they seem to want to support EFI which in my eyes is a huge plus.

I would appreciate any ideas or suggestions that might help me achieve
my goal. Most importantly, am I perhaps being too aggressive in looking
for ARMv9? Should I settle for something less recent?

Best,
Christian

PS: while I'm currently still testing manually, my goal is to set up a
CI for this, so my packages get continuous testing.