Bug#728759: linux - compat_sys_getrlimit returns values compat_sys_setrlimit does not like
Package: src:linux
Version: 3.2.51-1
Severity: normal
compat_sys_getrlimit clips all values above COMPAT_RLIM_INFINITY. So
any large values between COMPAT_RLIM_INFINITY and RLIM_INFINITY is
returned as the same value COMPAT_RLIM_INFINITY.
compat_sys_setrlimit converts COMPAT_RLIM_INFINITY to RLIM_INFINITY.
This new value will be not accepted as it is higher then the old value.
Combined with a wrong value in glibc on mips* this leads to
| getrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=RLIM_INFINITY}) = 0
| setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=RLIM_INFINITY}) = -1 EPERM (Operation not permitted)
All other architectures should be also affected if the 32bit- (setrlimit,
getrlimit, prlimit) and 64bit-interface (prlimit64) is mixed.
Bastian
-- System Information:
Debian Release: 7.2
APT prefers stable
APT policy: (990, 'stable'), (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: i386 (i686)
Kernel: Linux 3.10-3-686-pae (SMP w/2 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Reply to: