Bug#559035: Random segfaults with 2.6.26 on amd64
Further investigation showed that process address space randomization
in 2.6.26-2-xen-amd64 and 2.6.26-2-amd64 is causing random segfaults
in exim4 and python. Problem can be reproduced by starting exim4 in
a loop:
# ulimit -c unlimited
# for ((i = 0; i <= 100000; i++)); do exim4 -bV || break; done > /dev/null; echo $i
Segmentation fault
12652
The segfaults stop if address space randomization is disabled or stack
size increased:
# echo 0 > /proc/sys/kernel/randomize_va_space
# for ((i = 0; i <= 100000; i++)); do exim4 -bV || break; done > /dev/null; echo $i
100001
# echo 2 > /proc/sys/kernel/randomize_va_space
# ulimit -s unlimited
# for ((i = 0; i <= 100000; i++)); do exim4 -bV || break; done > /dev/null; echo $i
100001
Core file generated during segfault shows that the stack segment got
allocated in-between code segments and the application fails when it
tries to grow the stack, but instead accesses a read-only segment:
# gdb /usr/sbin/exim4 core
(gdb) p $rsp
$10 = (void *) 0x7fff249dd160
(gdb) maintenance info sections
Exec file:
`/usr/sbin/exim4', file type elf64-x86-64.
0x0000000000400200->0x000000000040021c at 0x00000200: .interp ALLOC LOAD READONLY DATA HAS_CONTENTS
0x000000000040021c->0x000000000040023c at 0x0000021c: .note.ABI-tag ALLOC LOAD READONLY DATA HAS_CONTENTS
0x0000000000400240->0x0000000000400b2c at 0x00000240: .hash ALLOC LOAD READONLY DATA HAS_CONTENTS
0x0000000000400b30->0x0000000000400dc0 at 0x00000b30: .gnu.hash ALLOC LOAD READONLY DATA HAS_CONTENTS
0x0000000000400dc0->0x0000000000402a70 at 0x00000dc0: .dynsym ALLOC LOAD READONLY DATA HAS_CONTENTS
0x0000000000402a70->0x00000000004038e3 at 0x00002a70: .dynstr ALLOC LOAD READONLY DATA HAS_CONTENTS
0x00000000004038e4->0x0000000000403b48 at 0x000038e4: .gnu.version ALLOC LOAD READONLY DATA HAS_CONTENTS
0x0000000000403b48->0x0000000000403c58 at 0x00003b48: .gnu.version_r ALLOC LOAD READONLY DATA HAS_CONTENTS
0x0000000000403c58->0x0000000000403d18 at 0x00003c58: .rela.dyn ALLOC LOAD READONLY DATA HAS_CONTENTS
0x0000000000403d18->0x0000000000405248 at 0x00003d18: .rela.plt ALLOC LOAD READONLY DATA HAS_CONTENTS
0x0000000000405248->0x0000000000405260 at 0x00005248: .init ALLOC LOAD READONLY CODE HAS_CONTENTS
0x0000000000405260->0x0000000000406090 at 0x00005260: .plt ALLOC LOAD READONLY CODE HAS_CONTENTS
0x0000000000406090->0x0000000000482cc8 at 0x00006090: .text ALLOC LOAD READONLY CODE HAS_CONTENTS
0x0000000000482cc8->0x0000000000482cd6 at 0x00082cc8: .fini ALLOC LOAD READONLY CODE HAS_CONTENTS
0x0000000000482ce0->0x00000000004a5fd8 at 0x00082ce0: .rodata ALLOC LOAD READONLY DATA HAS_CONTENTS
0x00000000004a5fd8->0x00000000004a7244 at 0x000a5fd8: .eh_frame_hdr ALLOC LOAD READONLY DATA HAS_CONTENTS
0x00000000004a7248->0x00000000004acc94 at 0x000a7248: .eh_frame ALLOC LOAD READONLY DATA HAS_CONTENTS
0x00000000006ad000->0x00000000006ad010 at 0x000ad000: .ctors ALLOC LOAD DATA HAS_CONTENTS
0x00000000006ad010->0x00000000006ad020 at 0x000ad010: .dtors ALLOC LOAD DATA HAS_CONTENTS
0x00000000006ad020->0x00000000006ad028 at 0x000ad020: .jcr ALLOC LOAD DATA HAS_CONTENTS
0x00000000006ad028->0x00000000006ad248 at 0x000ad028: .dynamic ALLOC LOAD DATA HAS_CONTENTS
0x00000000006ad248->0x00000000006ad250 at 0x000ad248: .got ALLOC LOAD DATA HAS_CONTENTS
0x00000000006ad250->0x00000000006ad978 at 0x000ad250: .got.plt ALLOC LOAD DATA HAS_CONTENTS
0x00000000006ad980->0x00000000006b388c at 0x000ad980: .data ALLOC LOAD DATA HAS_CONTENTS
0x00000000006b38a0->0x00000000006b6f08 at 0x000b388c: .bss ALLOC
0x0000000000000000->0x000000000000000c at 0x000b388c: .gnu_debuglink READONLY HAS_CONTENTS
Core file:
`/root/core', file type elf64-x86-64.
0x0000000000000000->0x0000000000000324 at 0x00000dd0: note0 READONLY HAS_CONTENTS
0x0000000000000000->0x00000000000000d8 at 0x00000e54: .reg/19565 HAS_CONTENTS
0x0000000000000000->0x00000000000000d8 at 0x00000e54: .reg HAS_CONTENTS
0x0000000000000000->0x0000000000000110 at 0x00000fe4: .auxv HAS_CONTENTS
0x0000000000400000->0x00000000004ad000 at 0x00002000: load1 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00000000006ad000->0x00000000006b4000 at 0x000af000: load2 ALLOC LOAD HAS_CONTENTS
0x00000000006b4000->0x00000000006b7000 at 0x000b6000: load3 ALLOC LOAD HAS_CONTENTS
0x0000000000a46000->0x0000000000a67000 at 0x000b9000: load4 ALLOC LOAD HAS_CONTENTS
0x00007fff23a2c000->0x00007fff23a91000 at 0x000da000: load5 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff23a91000->0x00007fff23c90000 at 0x0013f000: load6 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff23c90000->0x00007fff23c93000 at 0x0033e000: load7 ALLOC LOAD HAS_CONTENTS
0x00007fff23c93000->0x00007fff23ca9000 at 0x00341000: load8 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff23ca9000->0x00007fff23ea9000 at 0x00357000: load9 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff23ea9000->0x00007fff23eaa000 at 0x00557000: load10 ALLOC LOAD HAS_CONTENTS
0x00007fff23eaa000->0x00007fff23eb9000 at 0x00558000: load11 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff23eb9000->0x00007fff240b9000 at 0x00567000: load12 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff240b9000->0x00007fff240ba000 at 0x00767000: load13 ALLOC LOAD HAS_CONTENTS
0x00007fff240ba000->0x00007fff24204000 at 0x00768000: load14 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff24204000->0x00007fff24403000 at 0x008b2000: load15 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff24403000->0x00007fff24406000 at 0x00ab1000: load16 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff24406000->0x00007fff24408000 at 0x00ab4000: load17 ALLOC LOAD HAS_CONTENTS
0x00007fff24408000->0x00007fff2440d000 at 0x00ab6000: load18 ALLOC LOAD HAS_CONTENTS
0x00007fff2440d000->0x00007fff244b4000 at 0x00abb000: load19 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff244b4000->0x00007fff246b4000 at 0x00b62000: load20 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff246b4000->0x00007fff246bf000 at 0x00d62000: load21 ALLOC LOAD HAS_CONTENTS
0x00007fff246bf000->0x00007fff24803000 at 0x00d6d000: load22 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff24803000->0x00007fff24a03000 at 0x00eb1000: load23 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff24a03000->0x00007fff24a08000 at 0x010b1000: load24 ALLOC LOAD HAS_CONTENTS
0x00007fff24a08000->0x00007fff24a08000 at 0x010b6000: load25 ALLOC
s-> 0x00007fff24a09000->0x00007fff24a1e000 at 0x010b6000: load26 ALLOC LOAD HAS_CONTENTS
0x00007fff24ab1000->0x00007fff24ab2000 at 0x010cb000: load27 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff24b34000->0x00007fff24b4a000 at 0x010cc000: load28 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff24b4a000->0x00007fff24d4a000 at 0x010e2000: load29 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff24d4a000->0x00007fff24d4c000 at 0x012e2000: load30 ALLOC LOAD HAS_CONTENTS
0x00007fff24d4c000->0x00007fff24d50000 at 0x012e4000: load31 ALLOC LOAD HAS_CONTENTS
0x00007fff24d50000->0x00007fff24d52000 at 0x012e8000: load32 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff24d52000->0x00007fff24f52000 at 0x012ea000: load33 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff24f52000->0x00007fff24f54000 at 0x014ea000: load34 ALLOC LOAD HAS_CONTENTS
0x00007fff24f54000->0x00007fff24fd6000 at 0x014ec000: load35 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff24fd6000->0x00007fff251d5000 at 0x0156e000: load36 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff251d5000->0x00007fff251d7000 at 0x0176d000: load37 ALLOC LOAD HAS_CONTENTS
0x00007fff251d7000->0x00007fff251df000 at 0x0176f000: load38 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff251df000->0x00007fff253df000 at 0x01777000: load39 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff253df000->0x00007fff253e1000 at 0x01977000: load40 ALLOC LOAD HAS_CONTENTS
0x00007fff253e1000->0x00007fff253e1000 at 0x01979000: load41 ALLOC
0x00007fff2540f000->0x00007fff25424000 at 0x01979000: load42 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff25424000->0x00007fff25623000 at 0x0198e000: load43 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff25623000->0x00007fff25625000 at 0x01b8d000: load44 ALLOC LOAD HAS_CONTENTS
0x00007fff25625000->0x00007fff25625000 at 0x01b8f000: load45 ALLOC
0x00007fff25627000->0x00007fff25637000 at 0x01b8f000: load46 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff25637000->0x00007fff25837000 at 0x01b9f000: load47 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff25837000->0x00007fff25839000 at 0x01d9f000: load48 ALLOC LOAD HAS_CONTENTS
0x00007fff25839000->0x00007fff25839000 at 0x01da1000: load49 ALLOC
0x00007fff2583b000->0x00007fff25863000 at 0x01da1000: load50 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff25863000->0x00007fff25a62000 at 0x01dc9000: load51 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff25a62000->0x00007fff25a63000 at 0x01fc8000: load52 ALLOC LOAD HAS_CONTENTS
0x00007fff25a63000->0x00007fff25a7f000 at 0x01fc9000: load53 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff25b68000->0x00007fff25b6b000 at 0x01fe5000: load54 ALLOC LOAD HAS_CONTENTS
0x00007fff25b6b000->0x00007fff25b6e000 at 0x01fe8000: load55 ALLOC LOAD READONLY CODE HAS_CONTENTS
0x00007fff25b6e000->0x00007fff25c6d000 at 0x01feb000: load56 ALLOC LOAD READONLY HAS_CONTENTS
0x00007fff25c6d000->0x00007fff25c6e000 at 0x020ea000: load57 ALLOC LOAD HAS_CONTENTS
0x00007fff25c6e000->0x00007fff25c72000 at 0x020eb000: load58 ALLOC LOAD HAS_CONTENTS
0x00007fff25c7b000->0x00007fff25c7e000 at 0x020ef000: load59 ALLOC LOAD HAS_CONTENTS
0x00007fff25c7e000->0x00007fff25c80000 at 0x020f2000: load60 ALLOC LOAD HAS_CONTENTS
0xffffffffff600000->0xffffffffff600000 at 0x020f4000: load61 ALLOC READONLY CODE
Segfault can't be reproduced on 2.6.31-1-amd64 from unstable and in fact
it was fixed in the mainline kernel few months ago:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=80938332d8cf652f6b16e0788cf0ca136befe0b5
After applying this patch to 2.6.26-2-xen-amd64 the segfaults don't
happen any more, so please include it in Debian kernel packages.
--
Valentin
Reply to: