[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: latency test results for armhf vs arm64?

On 7/16/22 08:10, Arnd Bergmann wrote:
On Sat, Jul 16, 2022 at 5:05 AM gene heskett <gheskett@shentel.net> wrote:
On 7/15/22 21:54, Paul Wise wrote:
On Fri, 2022-07-15 at 13:40 -0400, gene heskett wrote:

Because our latency-test results are better on armhf than on arm64,
we use armhf for its performance.
Are these results for armhf kernel with armhf userland?
The whole install of raspios is armhf. So I guess its yes.

Are the results for arm64 kernel with armhf userland similar?
I have not tried to build an aarch64 from the src I have.
How much worse are the results for arm64 kernel and userland?
No, not exact, but its roughly 4x longer when I can get it to run,
as it did check for a realtime preempt kernel and did a graceful
exit if not found. So its not been run on 64 bit Debian image since
There are unfortunately a number of variables here that make things
really hard to compare, any of these can have an effect that dominates
your results:

- 4.19 is four years old, and both the mainline kernel and the
   preempt-rt patches have changed a lot in the meantime. It's
   possible that a current preempt-rt has regressed compared to
   the version you are running. If so, we can work on fixing the
   regression for future kernels, but there won't be much interest
   in working on the old kernel

- Raspberry Pi OS (and Raspbian before that) has a number of
   platform specific kernel patches that are neither in mainline
   Linux nor in the Debian kernel packages. It is possible that they
   have already identified and fixed a source of latency in their
   kernels but not managed to upstream that fix for a number of

Their kernels are uniformly horrible with latency's ranging to above a millisecond.
Linuxcnc simply refuses to run on those kernels.
- A lot of kernel configuration options can have a huge impact
   on latency, it's not just preempt-rt that can be turned on or off,
   but any device driver that disables preemption for too long can
   increase the maximum latency of the system.

- The raspbian user space should have very little effect on
    latency but it's worth pointing out that you may see different
    performance between armv6 (raspbian)
I wish you would admit that the raspios I am running IS armhf (kernel7l)
I've no clue where you got the impression it was v6. It is not.
and armv7 (debian
    armhf), between vfpv3-d16 (raspbian and debian armhf) and
    neon-d32 (fedora and others), and between a32 (raspbian),
    t32 (debian armhf) and a64 (debian arm64), instruction sets
    running the same code. In most applications the effect is very
    small, and it's not always the same one that's fast either.

It, latency-test, has recently been worked on but I've not tested
it on a Debian image of either flavor since.
Do you have your latency-test output available for reference

To establish a baseline, it would be good if someone could run
the same test using debian armhf userland on similar hardware
with this kernel:

If that can reproduce the bad numbers you observed, the next
step would be to try a corresponding 32-bit kernel and see if
that is better, but that requires building a custom package.
There is a linux-image-rt-armmp package in bookworm, but to
get PCI and USB3 working on Raspberry Pi 4, one needs to
enable both the PCI driver and CONFIG_ARM_LPAE, possibly

It is now in testing so those of you w/o 5 years of history to lose
could try it for the price of a 64G u-sd card.
For some reason, linuxcnc is still missing for armhf. I managed
to build the source package, which had a minor issue finding the
libboost_python310 dependency, but it worked after I added
that. I don't have the right system to test on myself though.

[Side note: I hope you are not storing any important data on an
  SD card. Even with "industrial grade" ones, I would recommend
  doing regular backups to more permanent storage, and the
  usual consumer cards are not designed to handle running a
  general-purpose OS at all and will cause data corruption over
The card its running on right now is over 2 years old, zero problems,
and has had all updates, including a daily update of linuxcnc from the
the buildbot, or if the buildbot is down, my own scripts also building
installable deb's. The secret is use a big enough card that it has enough
room to do its maintenance. 64G card has around 15G's on it.

Take care and stay well everybody.


Cheers, Gene Heskett.
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/>

Reply to: