Agenda for todays LSB Test telco
All
Enclosed is an agenda for todays LSB test teleconference
regards
Andrew
Agenda
----------
1. Introductions
2. Moving Current test tools to 1.0 status by 2001-12-31
source testsets currently in beta
VSX-PCTS
LSB-OS
LSB-FHS
LSB-USRGROUPS
What needs to be done for these?
Status of other tools?
3. Bug Analysis from LSB results database
Below are the 9 LSB-OS and 5 LSB-PCTS test currently classified
as possible testcase failures in the LSB test results database.
With them is an analysis from our engineers with a recommendation
as to status.
The searchable database is at
http://www.freestandards.org/lsb/test/results/
The raw journal data for the test results are available for from:
ftp://ftp.freestandards.org/pub/lsb/test_results/2001-08-15/
We need to review these and agree a course for resolution
of these.
>
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/LSB.os/devclass/grantpt_L/T.grantpt_L-2
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> grantpt(5) did not give correct results
> ERRNO VALUES: expected: 9 (EBADF), observed: 25 (ENOTTY)
>
> ----------------------
> User: raj
> Time (GMT): 01 Nov 2001 16:43:54
> Change cause from Unknown to Testcase
>
> This looks like a wrongly reported error condition
> See the LSB-OS beta deliverable (0.7 March 2001)
> /tset/LSB.os/devclass/grantpt_L/T.grantpt_L{2,3}
>
> These look like wrongly reported error conditions.
The test is correct. XSH5 specifies EBADF when the fd is not open.
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/LSB.os/devclass/grantpt_L/T.grantpt_L-3
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> grantpt(5) did not give correct results
> ERRNO VALUES: expected: 22 (EINVAL), observed: 25 (ENOTTY)
>
> ----------------------
> User: raj
> Time (GMT): 01 Nov 2001 16:42:48
> Change cause from Unknown to Testcase
>
> This look like a wrongly reported error condition
> See the LSB-OS beta deliverable (0.7 March 2001)
> /tset/LSB.os/devclass/grantpt_L/T.grantpt_L{2,3}
>
> These look like wrongly reported error conditions.
The test is correct. XSH5 specifies EINVAL when the fd is not
associated with a master pty. (In the test it is a plain file.)
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/LSB.os/ipc/shmdt/T.shmdt-2
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> -------START OLD ANALYSIS------------
> Original State: FAIL
> Original Cause: Testcase
> shmdt(addr) unexpected return with invalid address 0x805d1f0 in dynamically
> allocated data area
> RETURN VALUES: expected: -1, observed: 0
> ERRNO VALUES: expected: 22 (EINVAL), observed: 0 (NO ERROR)
> shmdt(addr) unexpected return with invalid addre
> ----------------------
> User: raj
> Time (GMT): 19 Oct 2001 20:48:34
> Change cause from Unknown to Testcase
>
> The function shmdt can fail only if there is no shared memory segment
> attached at addr, in such a case at return errno will be set to EINVAL
> -------END OLD ANALYSIS--------------
>
> shmdt(addr) unexpected return with invalid address 0x805d250 in dynamically
> allocated data area
> RETURN VALUES: expected: -1, observed: 0
> ERRNO VALUES: expected: 22 (EINVAL), observed: 0 (NO ERROR)
> shmdt(addr) unexpected return with invalid address 0x4001a004 not pointing
> to beginning of shared memory segment
> RETURN VALUES: expected: -1, observed: 0
> ERRNO VALUES: expected: 22 (EINVAL), observed: 0 (NO ERROR)
>
> Looks like the test should not fail in this condtion.
I'm not sure what to make of this comment. I agree that "the test
should not fail". I.e. if the system was compliant the test would
not fail.
If they want to claim this as a testcase bug, they need to provide
better rationale.
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/LSB.os/libdl/dlopen/T.dlopen-4
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> -------START OLD ANALYSIS------------
> Original State: FAIL
> Original Cause: Testcase
> dlopen(0, RTLD_NOW) did not allow access to symbol "test4_func" in the
> executable image:
> dlsym(handle2, "test4_func") failed :
> /home/tet/test_sets/TESTROOT/tset/LSB.os/libdl/dlopen/T.dlopen: undefined
> symbol: test4_func
>
> ----------------------
> User: raj
> Time (GMT): 22 Oct 2001 23:41:04
> Change cause from Unknown to Testcase
>
> \"test4_func\" is not defined
>
> -------END OLD ANALYSIS--------------
>
> dlopen(0, RTLD_NOW) did not allow access to symbol "test4_func" in the
> executable image:
> dlsym(handle2, "test4_func") failed :
> /home/tet/test_sets/TESTROOT/tset/LSB.os/libdl/dlopen/T.dlopen: undefined
> symbol: test4_func
test4_func() is defined at line 532 of dlopen.c so
this looks like an implementation issue
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/LSB.os/mfiles/msync_P/T.msync_P-1
> /tset/LSB.os/mfiles/msync_P/T.msync_P-2
> /tset/LSB.os/mfiles/msync_P/T.msync_P-3
> /tset/LSB.os/mfiles/msync_P/T.msync_P-4
> /tset/LSB.os/mfiles/msync_P/T.msync_P-5
>
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> chmod() failed, errno = 1 (EPERM)
>
> ----------------------
> User: lsarabia
> Time (GMT): 31 Oct 2001 21:30:21
> Change cause from Unknown to Testcase
>
> Failures relate to the fact that shm_open() returns ENOSYS.
> The msync_P failures occured because of a test suite set-up problem.
> See the LSB-OS beta deliverable (0.7 March 2001)
The comment "Failures relate to the fact that shm_open() returns ENOSYS"
is from the original failure analysis that we did, where it was a comment
on a larger group of tests and was preceded by "Most of these ..."
It didn't apply to msync_P.
The original analysis said "The msync_P failures occured because of a
test suite set-up problem. Rerunning these tests gave failures only
on tests 7 & 8". I'm afraid I don't know the details of what was
changed in the set-up to get them to work.
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/ANSI.os/streamio/tmpfile/T.tmpfile-2
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> tmpfile() did not creat file with mode 0666 when umask was set to 0, found
> 0600
>
> ----------------------
> User: raj
> Time (GMT): 01 Nov 2001 18:58:07
> Change cause from Unknown to Testcase
>
> tmpfile() ignores umask for group/other component
> This is intentional behaviour of glibc for reasons of security
> This test should be removed (or rewritten to expect current behaviour)
This is a well known glibc issue that has been discussed with Ulrich in
the past. It is possible to comply with POSIX and still be secure.
The method is outlined in the POSIX.1 interp on this issue - change the mode
using fchmod() after the file has been unlinked. For extra security it
could be improved by checking (with fstat()) that nobody created another
link to the file during the timing window before it was unlinked.
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/POSIX.hdr/misc/dirent/T.dirent-2
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> Compilation exited with non-zero value when expected to succeed
> Feature test macros: -D_POSIX_C_SOURCE=199506
> Compiler or run-time messages or results:
> cc02cs.c:11: storage size of `var' isn't known
> Compilation exited with non-zero value when expected to succeed
> Feature test macros: -D_POSIX_C_SOURCE=199506 -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> cc02cs.c:11: storage size of `var' isn't known
>
> ----------------------
> User: lsarabia
> Time (GMT): 02 Nov 2001 20:17:54
> Change cause from Unknown to Testcase
>
> Test did not compile correctly.
This one may need a POSIX.1 interpretation. The failure occurs because
type DIR is an incomplete type. It is not clear whether POSIX.1 allows
that. All it says is that type DIR "represents a directory stream".
Although it does say in the description of closedir(), "dirp may no
longer point to an accessible object of type DIR", implying that it
must be possible for an object of type DIR to exist, which I think may
not be possible if DIR is an incomplete type. It might be worth
asking a C language expert (e.g. Clive Feather) about that last point.
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/POSIX.hdr/misc/dirent/T.dirent-6
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> Compilation exited with non-zero value when expected to succeed
> Feature test macros: -D_POSIX_C_SOURCE=199506
> Compiler or run-time messages or results:
> cc06es.c: In function `dummy':
> cc06es.c:17: storage size of `deref' isn't known
> cc06es.c:28: arithmetic on pointer to an incomplete type
> cc06es.c:28: dereferencing pointer to incomplete type
> Compilation exited with non-zero value when expected to succeed
> Feature test macros: -D_POSIX_C_SOURCE=199506 -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> cc06es.c: In function `dummy':
> cc06es.c:17: storage size of `deref' isn't known
> cc06es.c:28: arithmetic on pointer to an incomplete type
> cc06es.c:28: dereferencing pointer to incomplete type
>
> ----------------------
> User: lsarabia
> Time (GMT): 02 Nov 2001 20:18:57
> Change cause from Unknown to Testcase
>
> Test did not compile properly.
Same issue as above, regarding type DIR.
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1, TurboLinux 7.0
> /tset/POSIX.hdr/misc/signal/T.signal-39
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> Compilation exited with non-zero value when expected to succeed
> Feature test macros: -D_POSIX_C_SOURCE=199506
> Compiler or run-time messages or results:
> In file included from cc39es.c:9:
> signal.h.D:1869: parse error before string constant
> signal.h.D:1869: warning: no semicolon at end of struct or union
> signal.h.D:1894: field `__schedparam' has incomplete type
> signal.h.D:10882: parse error before string constant
> signal.h.D:10882: warning: no semicolon at end of struct or union
> signal.h.D:10883: parse error before string constant
> Compilation exited with non-zero value when expected to succeed
> Feature test macros: -D_POSIX_C_SOURCE=199506 -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> In file included from cc39es.c:9:
> signal.h.D:1869: parse error before string constant
> signal.h.D:1869: warning: no semicolon at end of struct or union
> signal.h.D:1894: field `__schedparam' has incomplete type
> signal.h.D:10882: parse error before string constant
> signal.h.D:10882: warning: no semicolon at end of struct or union
> signal.h.D:10883: parse error before string constant
>
> ----------------------
> User: lsarabia
> Time (GMT): 02 Nov 2001 20:19:56
> Change cause from Unknown to Testcase
>
> Errors in test. parse error
This looks like a namespace pollution failure related to a structure
member. My guess is that the culprit is sched_priority in <bits/sched.h>,
which I believe has recently been changed to __sched_priority.
> =================================================================
> Fails on SuSE 7.2, Caldera 3.1, Debian 3.0, Red Hat 7.1
> /tset/ANSI.hdr/misc/stdio_2/T.stdio_2-41
>
> Caldera 3.1
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> Compilation exited with non-zero value when expected to succeed
> Feature test macros: -D_POSIX_C_SOURCE=199506
> Compiler or run-time messages or results:
> In file included from cc41es.c:9:
> stdio.h.D:4203: parse error before string constant
> stdio.h.D:4203: warning: no semicolon at end of struct or union
> stdio.h.D:4228: field `__schedparam' has incomplete type
> Compilation exited with non-zero value when expected to succeed
> Feature test macros: -D_POSIX_C_SOURCE=199506 -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> In file included from cc41es.c:9:
> stdio.h.D:4203: parse error before string constant
> stdio.h.D:4203: warning: no semicolon at end of struct or union
> stdio.h.D:4228: field `__schedparam' has incomplete type
>
> ----------------------
> User: lsarabia
> Time (GMT): 02 Nov 2001 20:41:33
> Change cause from Unknown to Testcase
>
> Errors in stdio.h.D within testcase.
See signal-39.
Note that stdio.h.D is not part of the test suite - it is generated
from the system's <stdio.h> header.
> ------------------------------------------------
>
>
> Debian 3.0
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> Feature test macros: -D_POSIX_C_SOURCE=199506
> Compiler or run-time messages or results:
> Illegal declared or 'typedef'ed symbol: popen
> Illegal declared or 'typedef'ed symbol: pclose
> Feature test macros: -D_POSIX_C_SOURCE=199506 -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> Illegal declared or 'typedef'ed symbol: popen
> Illegal declared or 'typedef'ed symbol: pclose
>
> ----------------------
> User: lsarabia
> Time (GMT): 02 Nov 2001 20:41:33
> Change cause from Unknown to Testcase
>
> Errors in stdio.h.D within testcase.
Bug in VSX4. (pclose and popen should be in the allowed symbols
list for POSIX96 mode.)
> --------------------------------------------------
>
>
> Red Hat 7.1
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> Feature test macros: -D_XOPEN_SOURCE=500
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=1
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=1
> -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=2
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=2
> -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=199309
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=199309
> -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=199506
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
> Feature test macros: -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=199506
> -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> Illegal #defined symbol: unix
>
> ----------------------
> User: lsarabia
> Time (GMT): 02 Nov 2001 20:41:33
> Change cause from Unknown to Testcase
>
> Errors in stdio.h.D within testcase.
As pointed out previously, stdio.h.D is not part of the test suite,
it is generated when the tests are built.
These failures are due to using gcc without the -ansi option. The
test suite requires an ANSI compiler, so either "gcc -ansi" or "c89"
(on systems that have it) must be used.
> ------------------------------------------------------
>
>
> SuSE 7.2
>
> Test result: FAIL
> Cause: Testcase
> Description:
>
> Feature test macros: -D_POSIX_C_SOURCE=199506
> Compiler or run-time messages or results:
> Illegal declared or 'typedef'ed symbol: popen
> Illegal declared or 'typedef'ed symbol: pclose
> Feature test macros: -D_POSIX_C_SOURCE=199506 -D_POSIX_SOURCE
> Compiler or run-time messages or results:
> Illegal declared or 'typedef'ed symbol: popen
> Illegal declared or 'typedef'ed symbol: pclose
>
> ----------------------
> User: lsarabia
> Time (GMT): 02 Nov 2001 20:41:33
> Change cause from Unknown to Testcase
>
> Errors in stdio.h.D within testcase.
See comment for Debian 3.0.
Reply to: