Re: Bug#625828: libipc-sharelite-perl: FTBFS on armel: test failures

[Niko Tyni <ntyni@debian.org>]

On Sun, May 08, 2011 at 09:43:55AM +0300, Niko Tyni wrote:
> On Fri, May 06, 2011 at 12:38:42PM +0300, Niko Tyni wrote:
> > Package: libipc-sharelite-perl
> > Version: 0.17-1
> > Severity: serious
> > Tags: wheezy sid
> > User: debian-perl@lists.debian.org
> > Usertags: perl-5.12-transition
> > 
> > This package failed to build on armel only against Perl 5.12.
> > 
> >  https://buildd.debian.org/status/fetch.php?pkg=libipc-sharelite-perl&arch=armel&ver=0.17-1%2Bb1&stamp=1304522076

> I can reproduce this on abel.debian.org with both squeeze (Perl 5.10)
> and sid (5.12), but not on agricola.debian.org at all. Either kernel or
> hardware specific? I see from the build log that arnold.debian.org (the
> buildd) is running "Linux 2.6.32 armel (armv5tel)" which matches abel.

It also failed on ancina.d.o ("Linux 2.6.31-rc9 armel (armv5tel)")
and alwyn.d.o ("Linux 2.6.32 armel (armv5tel)"). Dominic uploaded a
binary package built on agricola.d.o (2.6.26-2-iop32x), where it
never fails.

The code uses semaphore operations for locking shared memory, and the
bug seems to be that attaching the same shared memory segment twice and
then accessing the second copy breaks locking altogether.

I have narrowed the problem down to the attached pure C test case,
which fails almost all the time for me on abel.d.o but never anywhere
else I've tried.

I'm not intimately familiar with SysV IPC, but my reading of the semop(2)
and related manual pages indicates the code should work. I think this
looks like a kernel regression on armel, but I'm certainly happy to be
proved wrong.

@debian-arm: could some porters please test this (running it a dozen
times or so to see how consistent the result is) and report back with
the results and kernel versions? Please compile it without optimization,
otherwise the memory access triggering the bug gets optimized away.

abel% while   ! ./a.out 5000; do :; done
not ok 1 - got 5000, expected 10000
not ok 1 - got 8242, expected 10000
not ok 1 - got 9342, expected 10000
not ok 1 - got 5000, expected 10000
not ok 1 - got 7159, expected 10000
not ok 1 - got 5000, expected 10000
not ok 1 - got 9223, expected 10000
ok 1 - got 10000, expected 10000

Thanks,
-- 
Niko Tyni   ntyni@debian.org

/*
  Test case for http://bugs.debian.org/625828
  20110514 Niko Tyni <ntyni@debian.org>

  Derived from IPC::ShareLite:

    Copyright 1998-2002, Maurice Aubrey <maurice@hevanet.com>. 
    All rights reserved.

    This module is free software; you may redistribute it and/or
    modify it under the same terms as Perl itself.
*/

#include <stdio.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/shm.h>
#include <sys/sem.h>

#define DATA_SIZE 100
#define MYKEY ((key_t)1981)
#ifndef COUNT
#define COUNT 1000
#endif

#define GET_EX_LOCK(A)    semop((A), &ex_lock[0],    3)
#define RM_EX_LOCK(A)     semop((A), &ex_unlock[0],  2)

static struct sembuf ex_lock[3] = {
  {1, 0, 0},                    /* wait for readers to finish */
  {2, 0, 0},                    /* wait for writers to finish */
  {2, 1, SEM_UNDO}              /* assert write lock */
};

static struct sembuf ex_unlock[2] = {
  {1, 0, 0},                    /* wait for readers to finish */
  {2, -1, ( SEM_UNDO | IPC_NOWAIT )}    /* remove write lock */
};

void fatal ( char *error ) {
    perror(error);
    exit(1);
}

int main (int argc, char **argv) {
    pid_t pid;
    int i;
    int *data;
    int status;
    int semid;
    int shmid;
    int *shmaddr;
    int *shmaddr2;
    int c;
    int count=1000;

    if (argc > 1) {
        count = atoi(argv[1]);
    }

    if ((data = malloc(DATA_SIZE)) == NULL)
        fatal("malloc");
    memset(data, 0, DATA_SIZE);

    /* semaphore for the locks */
    if ( ( semid = semget( MYKEY, 3, (IPC_CREAT | 0666)) ) < 0 )
        fatal("semget");

    /* find/allocate a shared memory segment */
    if ( ( shmid = shmget( MYKEY, DATA_SIZE, IPC_CREAT | 0666) ) < 0 )
        fatal("shmget");

    /* attach the segment to our address space */
    if (( shmaddr = ( int * ) shmat( shmid, NULL, 0)) == (void *) -1)
        fatal("first shmat");

    /* attach a second time (at a different address) */
    if (( shmaddr2 = ( int * ) shmat( shmid, NULL, 0)) == (void *) -1)
        fatal("second shmat");

    /* lock */
    if (GET_EX_LOCK(semid) < 0)
        fatal("get lock");

    /* initialize */
    memcpy( shmaddr, data, DATA_SIZE );

    /* 
     * access the second attached range
     * if this line is removed (or optimized out), the bug goes away
     */
    c = *shmaddr2;

    /* it doesn't matter if the second range is detached or not */
    if (shmdt(shmaddr2) < 0)
        fatal("shmdt");

    /* unlock */
    if (RM_EX_LOCK(semid) < 0)
        fatal("unlock");

    /* make two processes access the segment concurrently with locking */
    if ((pid = fork()) < 0)
        fatal("fork");
    for (i=0; i<count; i++) {
        if (GET_EX_LOCK(semid) < 0)
            fatal("get lock");

        /* read, modify, and write back */
        memcpy( data, ( int * ) shmaddr,  sizeof(int));
        /* Uncomment for verbose output */
        /* fprintf(stderr, "%s: %d\n", (pid ? "p" : "k"), *data); */
        (*data)++;
        memcpy( shmaddr, data, sizeof(int));

        if (RM_EX_LOCK(semid) < 0)
            fatal("unlock");
    }
    if (!pid) /* kid */
        exit(0);

    /* parent */
    if (waitpid(pid, &status, 0) < 0)
        fatal("waitpid");

    if (WEXITSTATUS(status) != 0)
        fatal("kid exited abnormally");

    /* one more lock, just in case */
    if (GET_EX_LOCK(semid) < 0)
        fatal("get lock");

    /* read */
    memcpy( data, ( int * ) shmaddr,  sizeof(int));

    if (RM_EX_LOCK(semid) < 0)
        fatal("unlock");

    printf("%sok 1 - got %d, expected %d\n", ((*data == 2 * count) ? "":"not "), *data, 2 * count);
    return *data == 2*count ? 0 : 1;
}