Re: Freeze of masternode during an Amazon rebuild

[Sylvestre Ledru <sylvestre@debian.org>]

On 04/07/2013 22:09, Lucas Nussbaum wrote:
> On 04/07/13 at 21:27 +0200, Sylvestre Ledru wrote:
>> On 04/07/2013 11:31, Sylvestre Ledru wrote:
>>> Hello,
>>>
>>> I just started a huge rebuild: 48000 packages (normal + clang 3.3 +
>>> openjdk 7).
>>>
>>> After rebuilding around 39 % of all of them, the masternode froze for a
>>> while and just segfault (while I was writing this email).
>>>
>>> I will split the build and retry but it might be worst having a look.
>> Even with a split (18852 packages), I just had it again :/
> 
> how many slaves? how many slots? did you change the max number of open
> files as I suggested yesterday in private mail? 
So, I tried with "ulimit -n 8192"
A normal rebuild (~18500 packages) worked but a bigger (= 29151
packages) failed with the same error [1].

I will try again by splitting it.

Sylvestre
[1]
./masternode:195: [BUG] Segmentation fault
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0005 p:---- s:0035 b:0035 l:000034 d:000034 CFUNC  :split
c:0004 p:0379 s:0031 b:0031 l:000030 d:000030 METHOD ./masternode:195
c:0003 p:0219 s:0016 b:0016 l:002398 d:000015 BLOCK  ./masternode:250
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:---- s:0002 b:0002 l:000001 d:000001 TOP

-- Ruby level backtrace information ----------------------------------------
./masternode:250:in `block (3 levels) in <main>'
./masternode:195:in `execute_one_task'
./masternode:195:in `split'

-- C level backtrace information -------------------------------------------
/usr/lib/libruby-1.9.1.so.1.9(+0x158379) [0x7f875847d379]
/usr/lib/libruby-1.9.1.so.1.9(+0x5a4d9) [0x7f875837f4d9]
/usr/lib/libruby-1.9.1.so.1.9(rb_bug+0xb3) [0x7f875837fcc3]
/usr/lib/libruby-1.9.1.so.1.9(+0xf922f) [0x7f875841e22f]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf030) [0x7f8758118030]
/usr/lib/libruby-1.9.1.so.1.9(+0x1043b0) [0x7f87584293b0]
/usr/lib/libruby-1.9.1.so.1.9(rb_str_subseq+0x55) [0x7f8758438665]
/usr/lib/libruby-1.9.1.so.1.9(+0x115024) [0x7f875843a024]
/usr/lib/libruby-1.9.1.so.1.9(+0x152393) [0x7f8758477393]
/usr/lib/libruby-1.9.1.so.1.9(+0x148229) [0x7f875846d229]
/usr/lib/libruby-1.9.1.so.1.9(+0x14d8b4) [0x7f87584728b4]
/usr/lib/libruby-1.9.1.so.1.9(+0x14e4d0) [0x7f87584734d0]
/usr/lib/libruby-1.9.1.so.1.9(+0x15e19f) [0x7f875848319f]
/usr/lib/libruby-1.9.1.so.1.9(+0x15e27e) [0x7f875848327e]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50) [0x7f875810fb50]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8757594a7d]