Package: libc6
Version: 2.3.5-8
I'd like to report a bug in glibc (or possibly its interaction
with the Linux kernel). I have verified the bug both on Debian
and on Scientific Linux (derived from Red Hat). The bug can be
reproduced using the attached tarball:
tar zxvf BUG-glibc.tar.gz
cd BUG-glibc
make
The attached tarball is also available at:
http://www.ccs.neu.edu/home/gene/tmp/BUG-glibc.tar.gz
The details of the bug are in BUG-glibc/README in the tarball. In brief,
using setrlimit on RLIM_STACK, I set rlim_max to RLIM_INFINITY and rlim_cur
to something unnaturally high (e.g. 2 GB). I then call
system("./pthread"), where pthread is a program that calls
pthread_create(). pthread_create() then fails and the pthread process returns:pthread: Cannot allocate memory
pthread: pthread.c:22: main: Assertion `0 == pthread_retval' failed.
I have reproduced the bug on a Debian 2.6.8 kernel and on
a Red Hat 2.4.24 kernel. The hardware was an AMD Athlon 807 MHz processor
for one of the Debian kernels and an Intel(R) Xeon(TM) CPU 2.80GHz processor
for the Red Hat kernel.
Note that the bug does _not_ occur if I call pthread_create() directly from
my original process (in which I had called setrlimit() ). Note also
that the bug does _not_ occur if I invoke the original process using the
debugging version of the C library from libc6-dbg :
LD_LIBRARY_PATH=/usr/lib/debug ./a.out
does not return an error.
As a workaround, I can call setrlimit to set rlim_cur to a more reasonable
value for RLIM_STACK in the child process, "pthread", before invoking
pthread_create(). This avoids the bug.
Note also that in the transition from Linux kernel 2.2 to 2.4, there
was an issue when setrlimit was altered to use "unsigned int" instead
of "int". That discussion is available at:
http://www.ussg.iu.edu/hypermail/linux/kernel/0203.2/0716.html
and related links. I do not know if the current bug is related,
but I can trigger the bug on one machine by setting rlim_cur
to only: 0x5bee5006 (about 1.6 GB)
Attachment:
BUG-glibc.tar.gz
Description: application/tar-gz