Results for s390 (was Re: LSB 3.1 status for etch)
On Tue, 2006-08-01 at 10:50 -0400, Jeff Licquia wrote:
> Anyone willing to run the tests themselves can contact me for
> assistance. Or, if you're willing to give me root access on your
> machine to run the tests for you, please contact me.
And I have been contacted; thanks to all of you.
One person, Frans Pop, has kindly pointed me to hercules, the s390
emulator, and has run the tests under it. These results can tell us a
lot about the platform, but can also be misleading in other areas; to
what extent is the test finding bugs in the runtime vs. finding bugs in
the emulator? So, they should be looked at in that light.
I here post the results that are different from the i386 results I
posted earlier.
First of all, there are no results yet for vsw4 (the X11 test) or
desktop. This is due to the xvfb package being missing from etch for
s390. Strictly speaking, the LSB does not require that runtimes provide
xvfb, but it does make running the tests easier. I have not yet looked
into finding an appropriate xvfb (perhaps the sarge package, if one
exists; the xvfb from the LSB sample implementation might also work).
Here are the results we do have:
>From "lsb-runtime-test":
/tset/ANSI.os/genuts/abort/T.abort 11 UNREPORTED
/tset/ANSI.os/genuts/exit/T.exit 5 UNREPORTED
/tset/LSB.os/ipc/shmctl/T.shmctl 15 UNREPORTED
Frans did report that there were occasions where the test appeared to
hang; in at least one of these cases, the test had been running for 20
hours, which is excessive. All of these results are marked in the
journal as coming from a SIGTERM applied to the test.
/tset/ANSI.os/maths/exp/T.exp 1 FAIL
/tset/ANSI.os/maths/pow/T.pow 1 FAIL
These appear to be floating point precision errors. Here, for example,
are the journal results for the T.pow test:
400|128 1 1 12:52:36|IC Start
200|128 1 12:52:36|TP Start
520|128 1 00002597 1 1|Random arguments were tested from the interval [0.0941638, 10.6691]
520|128 1 00002597 1 2|The result was too large 368 times
520|128 1 00002597 1 3| equal 1281 times
520|128 1 00002597 1 4| too small 359 times
520|128 1 00002597 1 5| an ERANGE error 42 times
520|128 1 00002597 1 6|The maximum relative error of inf occured for values 1, 6.05487e+10
520|128 1 00002597 1 7|This gave a maximum loss of inf significant digits of base 2
520|128 1 00002597 1 8|The maximum acceptable loss is 80 significant digits
520|128 1 00002597 1 9|The root-mean-square relative error is 7.62839e+152
520|128 1 00002597 1 10|This gave an average loss of 5.6e+02 significant digits
of base 2
520|128 1 00002597 1 11|The maximum acceptable loss is 40 significant digits
520|128 1 00002597 1 12|The percentage of ERANGE errors is 2
520|128 1 00002597 1 13|The maximum acceptable percentage is 20
220|128 1 1 12:52:37|FAIL
410|128 1 1 12:52:37|IC End
I don't know to what extent these are bugs in the emulator.
>From "libchk":
libGL.so.1 1 FAIL
This was caused by a missing OpenGL library.
>From the C++ tests:
/24_iterators/istreambuf_iterator/2.cc 1 FAIL
/demangle/abi_examples/01.cc 1 FAIL
/demangle/abi_examples/02.cc 1 FAIL
This is actually a correction to the C++ tests on i386. I neglected to
report these results due to an oversight. All of these are problems in
the test suite and are already waived.
/22_locale/locale/cons/12658_thread.cc 1 FAIL
/thread/pthread4.cc 1 FAIL
The thread tests are always sensitive to running under an emulator, so
I'm hesitant to say for sure whether these are real failures.
Reply to: