Bug#302478: linux-kernel-headers: [asm/ioctl.h] _IOC_TYPECHECK trick fails under g++-3.4/4.0

[Jeremy Nimmer <jwnimmer@alum.mit.edu>]

Package: linux-kernel-headers
Version: 2.5.999-test7-bk-17
Severity: normal
Tags: patch

/usr/include/asm/ioctl.h employs an _IOC_TYPECHECK trick to catch mis-uses
of _IOR, _IOW, and _IORW, as discussed in http://lwn.net/Articles/48354/

Unfortunately, the trick is incompatible with switch statments as compiled
by g++ version 3.4 and up.  See ioctl.cc attached below.

$ g++-3.4 --version | head -1
g++-3.4 (GCC) 3.4.4 20050314 (prerelease) (Debian 3.4.3-12)
$ g++-3.4 -Wall ioctl.cc
ioctl.cc: In function `int main(int, char**)':
ioctl.cc:6: error: `__invalid_size_argument_for_IOC' cannot appear in a constant-expression

$ g++-4.0 --version | head -1
g++-4.0 (GCC) 4.0.0 20050326 (prerelease) (Debian 4.0-0pre9)
$ g++-4.0 -Wall ioctl.cc
ioctl.cc: In function 'int main(int, char**)':
ioctl.cc:6: error: '__invalid_size_argument_for_IOC' cannot appear in a constant-expression

The compilation succeeds with g++-3.3 and earlier, and also succeeds if
ioctl.cc is compiled with any gcc (as a C file).

This problem prevents use of any ioctl request constants in C++ switch
statements, which hinders (for example) writing C++ stubs or testing
machinery for ioctl-based interfaces.

Since the trick is meant to catch dangerous uses of _IOR/W in kernel
modules, I suggest that userland code not be exposed to it.  The
attached patch enables the trick iff __KERNEL__ is defined.

Thanks,
- Jeremy

-- System Information:
Debian Release: 3.1
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)
Kernel: Linux 2.4.26-1-686
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

-- no debconf information

--- ioctl.h.orig	2005-03-31 23:37:51.000000000 +0000
+++ ioctl.h	2005-03-31 23:39:05.000000000 +0000
@@ -52,12 +52,16 @@
 	 ((nr)   << _IOC_NRSHIFT) | \
 	 ((size) << _IOC_SIZESHIFT))
 
+#ifdef __KERNEL__
 /* provoke compile error for invalid uses of size argument */
 extern unsigned int __invalid_size_argument_for_IOC;
 #define _IOC_TYPECHECK(t) \
 	((sizeof(t) == sizeof(t[1]) && \
 	  sizeof(t) < (1 << _IOC_SIZEBITS)) ? \
 	  sizeof(t) : __invalid_size_argument_for_IOC)
+#else
+#define _IOC_TYPECHECK(t) (sizeof(t))
+#endif
 
 /* used to create numbers */
 #define _IO(type,nr)		_IOC(_IOC_NONE,(type),(nr),0)

#include <asm/ioctl.h>

#define TEST_IOCTL _IOR('T', 1, int )

int main(int argc, char **argv) {
	switch (argc) { case TEST_IOCTL: return 1; }
	return 0;
}