vsx-pcts tests for binary test suite
As part of the process of putting together a binary test suite package
we need to decide what to do with the tests that are failing (ie
remove/modify test or possibly fix behaviour).
I recently ran the test suite against an x86 system running 2.4.4 and
glibc 2.2.3. The following is a list of most of the tests that
reported failed, unresovled or warning (do we need to worry about
these?) and suggestions on how we treat each one.
Test which fail:
/tset/ANSI.os/maths/pow/T.pow 8
This test should remain. On x86 test system pow(double(1.0), NaN)
returns 1.0. Should be reported to the glibc folks to be fixed.
/tset/ANSI.os/streamio/tmpfile/T.tmpfile 2
tmpfile() ignores umask for group/other component. This is intentional
behaviour of glibc for reasons of security. This test should be
removed (or rewritten to expect current behaviour).
/tset/POSIX.os/dataform/cpio/T.cpio 26
/tset/POSIX.os/dataform/tar/T.tar 5
/tset/POSIX.os/dataform/tar/T.tar 6
/tset/POSIX.os/dataform/tar/T.tar 8
The above tests only fail if cpio/tar are used for the tests instead
of pax. I propose that these we do not require he use of pax and
remove these tests.
/tset/POSIX.os/dataform/tar/T.tar 9
"devmajor" field "" is not a zero-filled octal number
"devminor" field "" is not a zero-filled octal number
This test fails if tar or pax are used. Propose to remove/waiver
this test.
/tset/POSIX.os/devclass/c_iflag/T.c_iflag 19
/tset/POSIX.os/devclass/c_lflag/T.c_lflag 6
/tset/POSIX.os/devclass/tcdrain/T.tcdrain 2
These fail only if pseudo ttys are used during testing
instead of a loopback serial cable. Propose we remove these tests.
/tset/POSIX.os/devclass/tcsetpgrp/T.tcsetpgrp 6
tcsetpgrp(fd, 0) gave errno 1 (EPERM), expected EINVAL
No known fix available for this. May need to accept that this will
fail?
/tset/POSIX.os/files/unlink/T.unlink 19
Attempt to unlink a directory returns EISDIR instead of EPERM. Easy to
fix, but maybe hard to get a patch in since EISDIR kind of makes more
sense. Suggest we accept that EISDIR will be returned.
/tset/POSIX.os/ioprim/fcntl/T.fcntl 37
fcntl(6, F_DUPFD, 1024) did not return expected values
ERRNO VALUES: expected: 22 (EINVAL), observed: 24 (EMFILE)
No known patch available though likely not hard to fix.
/tset/POSIX.os/ioprim/read/T.read 13
time stamps incorrect: atime was not updated
time stamps incorrect: atime was not updated
Introduced between 2.2 and 2.4 kernels. atime of
pipes/fifos not updates on read
/tset/POSIX.os/procenv/sysconf/T.sysconf 10
sysconf(_SC_TZNAME_MAX) failed expected >= 6 or -1, got 3
Need to report to glibc folks?
Given this basically correlates header file to runtime value can this
test be removed (since for the binary test suites we're not testing
header files)?
/tset/POSIX.os/procprim/sigconcept/T.sigconcept 35
These fail only if pseudo ttys are used during testing
instead of a loopback serial cable. Propose we remove these tests.
Test which report unresolved:
/tset/POSIX.os/devclass/i_close/T.i_close 1
/tset/POSIX.os/devclass/i_spchars/T.i_spchars 1
/tset/POSIX.os/devclass/i_spchars/T.i_spchars 3
/tset/POSIX.os/devclass/i_spchars/T.i_spchars 5
/tset/POSIX.os/devclass/i_spchars/T.i_spchars 14
/tset/POSIX.os/devclass/i_spchars/T.i_spchars 17
/tset/POSIX.os/devclass/tcdrain/T.tcdrain 7
/tset/POSIX.os/devclass/tcflow/T.tcflow 2
/tset/POSIX.os/devclass/tcflow/T.tcflow 6
/tset/POSIX.os/devclass/tcflow/T.tcflow 7
/tset/POSIX.os/devclass/tcflow/T.tcflow 8
/tset/POSIX.os/devclass/tcflow/T.tcflow 13
/tset/POSIX.os/devclass/tcflow/T.tcflow 14
/tset/POSIX.os/devclass/tcflow/T.tcflow 15
/tset/POSIX.os/devclass/tcflow/T.tcflow 16
/tset/POSIX.os/devclass/tcflush/T.tcflush 1
/tset/POSIX.os/devclass/tcflush/T.tcflush 2
/tset/POSIX.os/devclass/tcflush/T.tcflush 3
/tset/POSIX.os/devclass/tcflush/T.tcflush 4
/tset/POSIX.os/devclass/tcflush/T.tcflush 5
/tset/POSIX.os/devclass/tcflush/T.tcflush 6
/tset/POSIX.os/devclass/tcflush/T.tcflush 10
/tset/POSIX.os/devclass/tcflush/T.tcflush 11
/tset/POSIX.os/devclass/tcflush/T.tcflush 12
/tset/POSIX.os/devclass/tcflush/T.tcflush 13
/tset/POSIX.os/devclass/tcsetattr/T.tcsetattr 1
/tset/POSIX.os/devclass/tcsetattr/T.tcsetattr 13
/tset/POSIX.os/procprim/sigconcept/T.sigconcept 30
/tset/POSIX.os/procprim/sigconcept/T.sigconcept 31
/tset/POSIX.os/procprim/sigconcept/T.sigconcept 34
/tset/POSIX.os/procprim/sigconcept/T.sigconcept 36
/tset/POSIX.os/procprim/sigconcept/T.sigconcept 37
These fail only if pseudo ttys are used during testing
instead of a loopback serial cable. Propose we remove these tests.
/tset/POSIX.os/procenv/sysconf/T.sysconf 3
CLK_TCK - comment in header file says this is now obsolete?
Warnings:
/tset/ANSI.os/maths/pow/T.pow 4
pow(0.0, -1.0) gave
RETURN VALUES: expected: -inf, observed: inf
Bit Representation: expected value: \000\000\000\000\000\000\360\377
Bit Representation: observed value: \000\000\000\000\000\000\360\177
pow(0.0, -1.0) gave
RETURN VALUES: expected: -inf, observed: inf
Bit Representation: expected value: \000\000\000\000\000\000\360\377
Bit Representation: observed value: \000\000\000\000\000\000\360\177
/tset/ANSI.os/streamio/fopen/T.fopen 28
fopen() succeeded when FOPEN_MAX streams were already open
--
cyeoh@samba.org
Reply to: