Small update:
I managed to build a 6.10 kernel with gcc 14 now.
I'll do some more stress testing with it, but looks very stable so far.
My kernel config is attached.
It's much smaller than the Debian kernel config config-6.10.7-sparc64-smp, and now I'm quite convinced that it's really some options that cause the stability issues.
Finding them is big challenge, though...