CPU frequency scaling has to be set to a fixed frequency, e.g. to performance. Ondemand likely will cause audio glitches, that might not be visible as xruns.