Bug#34953: The elm problem

[Richard Braakman <dark@xs4all.nl>]

Package: libc6
Version: 2.1.1-0.1
Severity: important

I've spent the day figuring out what goes wrong with a glib2.0-compiled
elm on a glibc2.1 system.  I didn't find the bug, but I managed to
distill a simple test case:

#include <stdio.h>

void bail(char *where) {
    perror(where);
    exit(1);
}

int main(void) {
    FILE *test;

    if ((test = fopen("seektestfile", "w+")) == NULL)  bail("fopen");
    if (fputs("Test data", test) < 0)  bail("fputs");
    if (fseek(test, 0, 2) != 0)  bail("fseek 1");
    if (fseek(test, 0, 0) != 0)  bail("fseek 2");
    if (fflush(test) != 0)  bail("fflush");
    exit(0);
}

If this program is compiled on a glibc2.0 system and run on a glibc2.1
system, the fflush call will fail with EINVAL.  If the program is
compiled on a glibc2.1 system and run on a glibc2.1 system, it works
fine.  The reason can be found in the relevant part of the straces.

Trace of glibc2.1-compiled (good) version:
[...]
open("seektestfile", O_RDWR|O_CREAT|O_TRUNC, 0666) = 4
fstat(4, {st_mode=S_ISVTX|0502, st_size=0, ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40014000
write(4, "Test data", 9)                = 9
fstat(4, {st_mode=S_ISVTX|0502, st_size=0, ...}) = 0
_llseek(4, 0, {0}, SEEK_SET)            = 0
read(4, "Test data", 4096)              = 9
_llseek(4, 9, {9}, SEEK_SET)            = 0
_llseek(4, -9, {0}, SEEK_CUR)           = 0
[...]

Trace of glibc2.0-compiled version:
[...]
open("seektestfile", O_RDWR|O_CREAT|O_TRUNC, 0666) = 4
fstat(4, {st_mode=S_ISVTX|0502, st_size=0, ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x400140
00
write(4, "Test data", 9)                = 9
fstat(4, {st_mode=S_ISVTX|0502, st_size=0, ...}) = 0
_llseek(4, 0, {0}, SEEK_SET)            = 0
read(4, "Test data", 4096)              = 9
_llseek(4, 9, {9}, SEEK_SET)            = 0
_llseek(4, 4294967287, 0xbffff9e8, SEEK_CUR) = -1 EINVAL (Invalid argument)

In both cases the final line is from fflush (not from fseek, which simply
sets some pointers and does not invoke a system call).  The number
4294967287 is exactly -9 when interpreted as an unsigned long, so
I expect that the problem is somewhere in the big-files handling.

I failed to track the problem down inside glibc, but I hope that this
test case helps.

Richard Braakman