Re: recent armel/armhf kernels ENOMEM on mprotect, maxima/axiom FTBFS
On Tue, 2014-08-19 at 18:50 -0400, Camm Maguire wrote:
> mprotect failure: 0xd49000 305430528 : Cannot allocate memory
> sgc disabled: Cannot allocate memory
I've now repro'd this (on a system running 3.10-0.bpo.2-armmp). (it the
dumps me to some gcl prompt which I don't seem to be able to exit ;-))
Experimentally it seems like the benign change you referred to was the
addition of __stack_chk_guard=random_ulong()? I think the enablement of
stack protection is very far from benign in this context, since it plays
various memory management tricks and adds guard pages etc.
I don't know much about stack protector, but overriding
__stack_chk_guard in main() seems odd to me, I'd expect it to be done by
libc but perhaps that is because you are writing a language runtime? Are
you using some magic compiler or linker options?
I've tried to to write a simplified test case but my (rather too basic)
attempts don't seem to be doing the trick. TBH I think you are going to
need the assistance of someone who knows more about
gcc/glibc/stack-protector than me, right now I don't think the kernel
angle is the one most likely to produce fruit.
> and when run under strace -f will show the brk calls and mprotect calls
> which fail.
They seem to be quite large (hundreds of MB) compared to the other such
calls -- is that expected?
> Oddly enough, when run under gdb, something is done to the
> runtime environment which prevents the failure from occurring, a mystery
> to me.
FWIW I get a SEGV.
(gdb) bt
#0 0x0007f348 in memprotect_test () at sgbc.c:957
#1 0x00084258 in do_memprotect_test () at sgbc.c:1000
#2 memprotect_test_reset () at sgbc.c:1019
#3 0x000b271e in gcl_init_alloc (cs_start=<optimized out>) at alloc.c:1090
#4 0x00021914 in main (argc=1, argv=0xbefffd14, envp=0xbefffd1c) at main.c:357
The call to gcl_init_alloc is before "__stack_chk_guard=random_ulong()",
but these functions seem to relate to memprotect. Not sure what is going
on there, but perhaps the stack trace gives you a clue.
Ian.
Reply to: