Bug#281775: PTHREAD_CANCEL_DISABLE broken for NPTL

[Sascha Demetrio <sd@b-comp.de>]

Package: libc6
Version: 2.3.2.ds1-18
Severity: normal

Thread cancellation can't be disabled when running a 2.6.x kernel (i.e. 
when using the new NPTL threading).

Effect: A thread T1 will be cancelled if another thread T2 requests the 
cancellation via pthread_cancel(), even if T1 has cancellation disabled 
via pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, 0).

Workaround: Force libc6 to use the old LinuxThreads implementation by 
setting LD_ASSUME_KERNEL=2.4.19

The bug is already listed (and solved) in redhat's bugtracking system, 
see bug #112512 on bugzilla.redhat.com.

Demonstration:
--------
#include <stdio.h>
#include <pthread.h>

pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond = PTHREAD_COND_INITIALIZER;

void *thread_fn(void *self)
{
  pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, 0);
  pthread_mutex_lock(&lock);
  pthread_cond_signal(&cond);
  puts("checkpoint 1");
  /* The main thread will try to cancel the following call. */
  pthread_cond_wait(&cond, &lock);
  puts("checkpoint 3");
  pthread_cond_signal(&cond);
  pthread_mutex_unlock(&lock);
  return 0;
}

int main(void)
{
  pthread_t thread;
  pthread_mutex_lock(&lock);
  pthread_create(&thread, 0, thread_fn, 0);
  /* Wait for 'thread' to reach checkpoint 1. */
  pthread_cond_wait(&cond, &lock);
  /* Cancel the thread.  Note: 'thread' has disabled cancellation. */
  pthread_cancel(thread);
  puts("checkpoint 2");
  /* If the cancellation went through, then 'thread' will be cancelled
   * in a call to 'pthread_cond_wait()', i.e. it will re-acquire the
   * lock and terminate as soon as we release the lock in the main
   * thread -- which will happen with the following
   * 'pthread_cond_wait()'. */
  pthread_cond_signal(&cond);
  /* In case of the demonstrated bug, we will wait forever. */
  pthread_cond_wait(&cond, &lock);
  puts("checkpoint 4");
  pthread_mutex_unlock(&lock);
  pthread_join(thread, 0);
  puts("OK");
  return 0;
}
--------

/tmp $ gcc test.c -lpthread
/tmp $ LD_ASSUME_KERNEL=2.4.19 ./a.out
checkpoint 1
checkpoint 2
checkpoint 3
checkpoint 4
OK
/tmp $ ./a.out
checkpoint 1
checkpoint 2
(Program hangs.)

Tested on/with:
  Debian unstable
  Linux-2.6.8.1 ('Linus' kernel)
  glibc-2.3.2.ds1-18
  GCC 3.3.2