Re: Alpha recommendations
[warning: long benchmarking geek-out ahead. Hit 'd' now if this bores
you.]
On Thu, Oct 15, 1998 at 02:21:22PM -0400, Jeff Sturm wrote:
> Ah yes, benchmarks. People most often quote SPECint95/SPECfp95. That's
> only one view of Alpha performance, and a very narrow one at that...
You castigate SPECint for being a "narrow view of Alpha performance"
and then go on to quote BYTEmark??? SPECint consists of 8 tests,
including gcc, a lisp interpreter, and a go-playing program, among
others. If a machine runs the SPECint tests fast, it's probably going
to do fairly well on the kinds of things I'm interested in (compilers,
interpreters, AI...).
As I understand it, the BYTEmark tests are kernels, like strcpy() and
qsort() and the like (I haven't looked into them in depth). Many of
the BYTEmark tests fit in L2 cache. The compiler can make a _huge_
difference in BYTEmark results, because a single extra optimization
can result in a 10% or more change in the run time of such a simple
loop. BYTEmark strongly favors the Alpha, because of its large fast
caches and high clock rates, while real applications are often
bottlenecked on the memory bus. (In this regard SPECint is also
somewhat lacking, because some of its tests have a working set that
will fit completely inside a 4 or 8 MB cache.)
Of course any benchmark other than your own application is inherently
flawed, but FWIW SPECint is a much better benchmark than BYTEmark for
a large class of important problems.
Also, SPEC rules require that the reporter publish all of the specs
and results; see http://www.specbench.org/ for a database of reported
results. AFAIK BYTEmark has no such rules, and are often quoted
"bare".
> My point is that there is no such thing as a "typical" benchmark. People
> quote SPEC saying that Alpha is comparable to Intel on integer, and
> superior on fp, but that's just one data point, and may be a very poor
> representation of the real-life appliactions you'll be using.
Absolutely. But BYTEmark is almost certainly no better. Do a search
for "BYTEmark" on comp.benchmarks and comp.arch if you're curious for
more.
> More surprising is how all these benchmarks seem to focus on CPU
> performance. System performance can be measured by CPU, memory, I/O,
> network, video and other factors...
It's "hard" to set up a benchmark suite that covers all system
components. Again, benchmarking your own application is the best
solution.
> Anyway, forgive my ramblings, back to work now :)
Oh yeah. Me too.
-andy
--
Andy Isaacson adisaacs@mtu.edu adi@acm.org Fight Spam, join CAUCE:
http://www.csl.mtu.edu/~adisaacs/ http://www.cauce.org/
Reply to: