[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#785185: apache2-bin: mpm_worker: segfaults in modperl_interp_unselect()



Package: apache2
Severity: important

hi,

as requested this is the new bugreport for mentioned core dump
while using worker-mpm. This is what i get upon starting gdb.

Im sorry im not sure i can supply the full core dump as the
system runs our main customers OTRS and it seems the core
dump may include certain informations which are not supposed
to be public.

 > root@support:/var/cache/apache2/mod_cache_disk# gdb apache2 core 
 > GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
 > Copyright (C) 2014 Free Software Foundation, Inc.
 > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 > This is free software: you are free to change and redistribute it.
 > There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
 > and "show warranty" for details.
 > This GDB was configured as "x86_64-linux-gnu".
 > Type "show configuration" for configuration details.
 > For bug reporting instructions, please see:
 > <http://www.gnu.org/software/gdb/bugs/>.
 > Find the GDB manual and other documentation resources online at:
 > <http://www.gnu.org/software/gdb/documentation/>.
 > For help, type "help".
 > Type "apropos word" to search for commands related to "word"...
 > Reading symbols from apache2...Reading symbols from /usr/lib/debug//usr/sbin/apache2...done.
 > done.
 > 
 > warning: core file may not match specified executable file.
 > [New LWP 41765]
 > [New LWP 41763]
 > [New LWP 41784]
 > [New LWP 41783]
 > [New LWP 41782]
 > [New LWP 41781]
 > [New LWP 41780]
 > [New LWP 41779]
 > [New LWP 41778]
 > [New LWP 41764]
 > [New LWP 41777]
 > [New LWP 41776]
 > [New LWP 41761]
 > [New LWP 41775]
 > [New LWP 41760]
 > [New LWP 41774]
 > [New LWP 41785]
 > [New LWP 41773]
 > [New LWP 41772]
 > [New LWP 41771]
 > [New LWP 41770]
 > [New LWP 41769]
 > [New LWP 41768]
 > [New LWP 41767]
 > [New LWP 41766]
 > [New LWP 41758]
 > [New LWP 41762]
 > [Thread debugging using libthread_db enabled]
 > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
 > Core was generated by `/usr/sbin/apache2 -k start'.
 > Program terminated with signal SIGSEGV, Segmentation fault.
 > #0  0x00007f3ff9c7dac2 in modperl_interp_unselect () from /usr/lib/apache2/modules/mod_perl.so



----- Forwarded message from Stefan Fritsch <sf@sfritsch.de> -----

Date: Sun, 10 May 2015 22:20:52 +0200
From: Stefan Fritsch <sf@sfritsch.de>
To: Michael Ablassmeier <abi@grinser.de>, 779078@bugs.debian.org
Subject: Re: Bug#779078: apache2-bin: event mpm: child segfault in notify_suspend causes parent to exit during log rotation

Hi,

On Tuesday 05 May 2015 14:01:56, Michael Ablassmeier wrote:
> we recently migrated over an OTRS instance from another system to
> debian jessie and it appears we may have the same issue, sporadic
> segfauls, but we do not use the event_mpm but the default worker:
> 
>  [Tue May 05 13:02:19.929973 2015] [mpm_worker:notice] [pid
> 41460:tid 140289534666624] AH00292: Apache/2.4.10 (Debian)
> OpenSSL/1.0.1k mod_perl/2.0.9dev Perl/v5.20.2 configured --
> resuming normal operations

This can't really be the same bug, because the original issue was in 
mpm_event specific code.


>  root@support:/opt/otrs# grep -c Segmentation
> /var/log/apache2/error.log 36
> 
> find dump output attached. It seems the problem went away after
> switching to prefork worker.

I have trouble finding out which thread caused a crash in that dump 
and why. Do still have the core and see if gdb gave any interesting 
info when loading the core dump, before the bt command? And please 
open a new bug report for this one.

There is one thread in

Thread 1 (Thread 0x7f3ff08ce700 (LWP 41765)):
#0  0x00007f3ff9c7dac2 in modperl_interp_unselect () from 
/usr/lib/apache2/modules/mod_perl.so
No symbol table info available.
#1  0x00007f3ffd5819be in run_cleanups (cref=<optimized out>) at 
/tmp/buildd/apr-1.5.1/memory/unix/apr_pools.c:2352
        c = <optimized out>

This could mean that there are some thread safety issues in mod_perl. 
If this turns out to be the culprit, having a dump with an unstripped 
mod_perl would be nice. Though the bug report should go to mod_perl, 
then.

Cheers,
Stefan

----- End forwarded message -----
Thread 27 (Thread 0x7f3ff20d1700 (LWP 41762)):
#0  0x00007f3ffd07c50d in poll () at ../sysdeps/unix/syscall-template.S:81
No locals.
#1  0x00007f3ffd58b4db in poll (__timeout=5000, __nfds=1, __fds=0x7f3ff20d0750) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
No locals.
#2  apr_wait_for_io_or_timeout (f=f@entry=0x0, s=s@entry=0x7f3ff00c40a0, for_read=for_read@entry=1) at /tmp/buildd/apr-1.5.1/support/unix/waitio.c:51
        pfd = {fd = 17, events = 1, revents = 0}
        timeout = <optimized out>
#3  0x00007f3ffd58523a in apr_socket_recv (sock=sock@entry=0x7f3ff00c40a0, buf=buf@entry=0x7f3ff0079048 "L&Ó?I\310bL\021\034\020\264\343K\375\337o\027", len=len@entry=0x7f3ff20d0810) at /tmp/buildd/apr-1.5.1/network_io/unix/sendrecv.c:87
        rv = <optimized out>
        arv = <optimized out>
#4  0x00007f3ffd7a5fc1 in socket_bucket_read (a=0x7f3ff00c2108, str=0x7f3ff20d0808, len=0x7f3ff20d0810, block=<optimized out>) at /tmp/buildd/apr-util-1.5.4/buckets/apr_buckets_socket.c:36
        p = 0x7f3ff00c40a0
        buf = 0x7f3ff0079048 "L&Ó?I\310bL\021\034\020\264\343K\375\337o\027"
        rv = <optimized out>
        timeout = 139912133997408
#5  0x00007f3ffdea6fc1 in ap_core_input_filter (f=0x7f3ff00c1120, b=0x7f3ff00c10a0, mode=<optimized out>, block=APR_BLOCK_READ, readbytes=5) at core_filters.c:236
        e = 0x7f3ff00c2108
        rv = <optimized out>
        net = <optimized out>
        ctx = 0x7f3ff00c1278
        str = 0x0
        len = 8000
#6  0x00007f3ffdecb1ba in logio_in_filter (f=<optimized out>, bb=0x7f3ff00c10a0, mode=<optimized out>, block=<optimized out>, readbytes=<optimized out>) at mod_logio.c:140
        length = 1430824257
        status = <optimized out>
        cf = 0x7f3ff00c4678
#7  0x00007f3ff93a8a1a in reqtimeout_filter (f=0x7f3ff20d0750, bb=0x7f3ff00c10a0, mode=AP_MODE_READBYTES, block=APR_BLOCK_READ, readbytes=5) at mod_reqtimeout.c:303
        now = 20000000
        saved_sock_timeout = 5000000
        ccfg = 0x7f3ff00c1170
#8  0x00007f3ff8d7a2ea in bio_filter_in_read (bio=0x7f3fd4001650, in=0x7f3fd0011693 "\314?\177", inlen=5) at ssl_engine_io.c:471
        inl = 5
        inctx = 0x7f3ff00bf028
        block = APR_BLOCK_READ
#9  0x00007f3ff87fe16c in BIO_read () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
No symbol table info available.
#10 0x00007f3ff8b283a4 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#11 0x00007f3ff8b2967d in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#12 0x00007f3ff8b26994 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#13 0x00007f3ff8d791ce in ssl_io_input_read (inctx=0x7f3ff00bf028, 
    buf=0x7f3ff00bf070 "Action=AgentDashboard;Subaction=Element;Name=0100-TicketPendingReminder;Filter=Locked;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQq=0.01\r\nAccept-Language: de,en-US;q=0."..., 
    len=0x7f3ff20d0c60) at ssl_engine_io.c:591
        wanted = 8192
        bytes = 139912586981488
        rc = -516
#14 0x00007f3ff8d7b8eb in ssl_io_input_getline (len=0x7f3ff20d0c58, 
    buf=0x7f3ff00bf070 "Action=AgentDashboard;Subaction=Element;Name=0100-TicketPendingReminder;Filter=Locked;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQq=0.01\r\nAccept-Language: de,en-US;q=0."..., 
    inctx=0x7f3ff00bf028) at ssl_engine_io.c:709
        status = <optimized out>
        tmplen = 0
        buflen = 8192
        pos = 0x0
        offset = 0
#15 ssl_io_filter_input (f=0x7f3ff00c1078, bb=0x7f3ff00b88b0, mode=<optimized out>, block=<optimized out>, readbytes=<optimized out>) at ssl_engine_io.c:1388
        pos = <optimized out>
        status = 0
        inctx = 0x7f3ff00bf028
        start = 0x7f3ff00bf070 "Action=AgentDashboard;Subaction=Element;Name=0100-TicketPendingReminder;Filter=Locked;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQq=0.01\r\nAccept-Language: de,en-US;q=0."...
        len = 0
        is_init = <optimized out>
#16 0x00007f3ffde95e83 in ap_rgetline_core (s=0x7f3ff00b70d0, n=8192, read=0x7f3ff20d0d98, r=0x7f3ff00b70a0, fold=0, bb=0x7f3ff00b88b0) at protocol.c:229
        rv = <optimized out>
        e = <optimized out>
        bytes_handled = 0
        current_alloc = 0
        pos = <optimized out>
        last_char = 0x0
        do_alloc = <optimized out>
        saw_eos = 0
#17 0x00007f3ffde9818e in read_request_line (bb=<optimized out>, r=<optimized out>) at protocol.c:590
        rv = 5000
        ll = 0x0
        pro = 0x64 <error: Cannot access memory at address 0x64>
        major = 1
        minor = 0
        http = "\270B\f\360?"
        len = 139912819580414
        num_blank_lines = 0
        max_blank_lines = 100
        uri = 0x64 <error: Cannot access memory at address 0x64>
#18 ap_read_request (conn=0x7f3ff00c42b8) at protocol.c:960
        p = 0x7f3ff00b7028
        expect = 0x64 <error: Cannot access memory at address 0x64>
        access_status = -234027184
        tmp_bb = 0x7f3ff00b88b0
        csd = 0x64
        cur_timeout = 139912819580414
#19 0x00007f3ffdebed5e in ap_process_http_sync_connection (c=0x7f3ff00c42b8) at http_core.c:181
        r = <optimized out>
        cs = 0x0
        csd = 0x7f3ff00c40a0
        mpm_state = 1
#20 ap_process_http_connection (c=0x7f3ff00c42b8) at http_core.c:231
No locals.
#21 0x00007f3ffdeb5790 in ap_run_process_connection (c=0x7f3ff00c42b8) at connection.c:41
        pHook = <optimized out>
        n = 2
        rv = -1
#22 0x00007f3ffa0b851b in process_socket (bucket_alloc=<optimized out>, my_thread_num=<optimized out>, my_child_num=<optimized out>, sock=<optimized out>, p=<optimized out>, thd=<optimized out>) at worker.c:619
        current_conn = 0x7f3ff00c42b8
        conn_id = 139912587002552
        sbh = 0x7f3ff00c42b0
#23 worker_thread (thd=0x7f3ff20d0750, dummy=0x1) at worker.c:978
        process_slot = 0
        thread_slot = 2
        csd = 0x7f3ff00c40a0
        bucket_alloc = 0x0
        last_ptrans = 0x7f3ff00c40a0
        ptrans = 0x7f3ff00c4028
        is_idle = -267631944
#24 0x00007f3ffd3500a4 in start_thread (arg=0x7f3ff20d1700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3ff20d1700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912620611328, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912620611328, 3304053687223499098, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#25 0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 26 (Thread 0x7f3ffde42780 (LWP 41758)):
#0  0x00007f3ffd356add in read () at ../sysdeps/unix/syscall-template.S:81
No locals.
#1  0x00007f3ffdeb8647 in read (__nbytes=1, __buf=0x7ffc839aec23, __fd=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/unistd.h:44
No locals.
#2  ap_mpm_podx_check (pod=<optimized out>) at mpm_unix.c:535
        c = -2 '\376'
        fd = 7
        rc = <optimized out>
#3  0x00007f3ffa0b6b34 in child_main (child_num_arg=0) at worker.c:1341
        threads = 0x7f400888a7b0
        ts = 0x7f3ffddb8110
        thread_attr = 0x7f3ffddb8130
        start_thread_id = 0x7f3ffddb8190
#4  0x00007f3ffa0b96c5 in make_child (s=0x7f3ffde18de0, slot=0) at worker.c:1427
No locals.
#5  0x00007f3ffa0b9f55 in perform_idle_server_maintenance () at worker.c:1635
        free_slots = {0, 3, 6923952, 32576, 11, 0, -50220944, 32575, 6923952, 32576, -104341515, 32575, 114817344, 32576, -36843528, 32575, 114817344, 32576, -104340831, 32575, 3, 0, -36843528, 32575, 3, 0, -44597078, 32575, 0, 0, 20, 0}
        ps = 0x7ffc839aed64
#6  server_main_loop (remaining_children_to_start=<optimized out>) at worker.c:1761
        status = 255
        pid = {pid = -1, in = 0x7f3ffd342620 <main_arena>, out = 0x18, err = 0x7f3ff9ea8240 <perl_module>}
        i = -2086998684
        old_gen = 32576
        child_slot = -2086998684
        exitwhy = APR_PROC_EXIT
        processed_status = 0
#7  worker_run (_pconf=0x0, plog=0x7ffc839aed64, s=0x7ffc839aed64) at worker.c:1832
        rv = -512
#8  0x00007f3ffde91cae in ap_run_mpm (pconf=0x7f3ffde40028, plog=0x7f3ffde14028, s=0x7f3ffde18de0) at mpm_common.c:94
        pHook = <optimized out>
        n = 0
        rv = -1
#9  0x00007f3ffde8b1f3 in main (argc=3, argv=0x7ffc839af018) at main.c:777
        c = 0 '\000'
        error = 0xfffffffffffffe00 <error: Cannot access memory at address 0xfffffffffffffe00>
        process = 0x7f3ffde49118
        pconf = 0x7f3ffde40028
        plog = 0x7f3ffde14028
        ptemp = 0x7f3ffde16028
        pcommands = 0x7f3ffde1e028
        opt = 0x7f3ffde1e118
        mod = 0x7f3ffe0f01c0 <ap_prelinked_modules+64>
        opt_arg = 0x7f3ffde49028 "(\320\344\375?\177"
        signal_server = 0xfffffffffffffe00

Thread 25 (Thread 0x7f3febfff700 (LWP 41766)):
#0  0x00007f3ffd07c50d in poll () at ../sysdeps/unix/syscall-template.S:81
No locals.
#1  0x00007f3ffd58b4db in poll (__timeout=20674, __nfds=1, __fds=0x7f3febffe750) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
No locals.
#2  apr_wait_for_io_or_timeout (f=f@entry=0x0, s=s@entry=0x7f3ff00a30a0, for_read=for_read@entry=1) at /tmp/buildd/apr-1.5.1/support/unix/waitio.c:51
        pfd = {fd = 22, events = 1, revents = 0}
        timeout = <optimized out>
#3  0x00007f3ffd58523a in apr_socket_recv (sock=sock@entry=0x7f3ff00a30a0, buf=buf@entry=0x7f3ff0096048 "\026\003\003", len=len@entry=0x7f3febffe810) at /tmp/buildd/apr-1.5.1/network_io/unix/sendrecv.c:87
        rv = <optimized out>
        arv = <optimized out>
#4  0x00007f3ffd7a5fc1 in socket_bucket_read (a=0x7f3ff00a1248, str=0x7f3febffe808, len=0x7f3febffe810, block=<optimized out>) at /tmp/buildd/apr-util-1.5.4/buckets/apr_buckets_socket.c:36
        p = 0x7f3ff00a30a0
        buf = 0x7f3ff0096048 "\026\003\003"
        rv = <optimized out>
        timeout = 139912519084096
#5  0x00007f3ffdea6fc1 in ap_core_input_filter (f=0x7f3ff00a0120, b=0x7f3ff00a00a0, mode=<optimized out>, block=APR_BLOCK_READ, readbytes=5) at core_filters.c:236
        e = 0x7f3ff00a1248
        rv = <optimized out>
        net = <optimized out>
        ctx = 0x7f3ff00a0278
        str = 0x0
        len = 8000
#6  0x00007f3ffdecb1ba in logio_in_filter (f=<optimized out>, bb=0x7f3ff00a00a0, mode=<optimized out>, block=<optimized out>, readbytes=<optimized out>) at mod_logio.c:140
        length = 1430824256
        status = <optimized out>
        cf = 0x7f3ff00a3678
#7  0x00007f3ff93a8a1a in reqtimeout_filter (f=0x7f3febffe750, bb=0x7f3ff00a00a0, mode=AP_MODE_READBYTES, block=APR_BLOCK_READ, readbytes=5) at mod_reqtimeout.c:303
        now = 20674847
        saved_sock_timeout = 300000000
        ccfg = 0x7f3ff00a0170
#8  0x00007f3ff8d7a2ea in bio_filter_in_read (bio=0x7f3fc8001650, in=0x7f3fc4006f83 "\320?\177", inlen=5) at ssl_engine_io.c:471
        inl = 5
        inctx = 0x7f3ff009e028
        block = APR_BLOCK_READ
#9  0x00007f3ff87fe16c in BIO_read () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
No symbol table info available.
#10 0x00007f3ff8b283a4 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#11 0x00007f3ff8b2967d in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#12 0x00007f3ff8b26994 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#13 0x00007f3ff8d791ce in ssl_io_input_read (inctx=0x7f3ff009e028, buf=0x7f3ff009e070 "", len=0x7f3febffec60) at ssl_engine_io.c:591
        wanted = 8192
        bytes = 139912586846320
        rc = -516
#14 0x00007f3ff8d7b8eb in ssl_io_input_getline (len=0x7f3febffec58, buf=0x7f3ff009e070 "", inctx=0x7f3ff009e028) at ssl_engine_io.c:709
        status = <optimized out>
        tmplen = 0
        buflen = 8192
        pos = 0x0
        offset = 0
#15 ssl_io_filter_input (f=0x7f3ff00a0078, bb=0x7f3ff009d8b0, mode=<optimized out>, block=<optimized out>, readbytes=<optimized out>) at ssl_engine_io.c:1388
        pos = <optimized out>
        status = 0
        inctx = 0x7f3ff009e028
        start = 0x7f3ff009e070 ""
        len = 0
        is_init = <optimized out>
#16 0x00007f3ffde95e83 in ap_rgetline_core (s=0x7f3ff009c0d0, n=8192, read=0x7f3febffed98, r=0x7f3ff009c0a0, fold=0, bb=0x7f3ff009d8b0) at protocol.c:229
        rv = <optimized out>
        e = <optimized out>
        bytes_handled = 0
        current_alloc = 0
        pos = <optimized out>
        last_char = 0x0
        do_alloc = <optimized out>
        saw_eos = 0
#17 0x00007f3ffde9818e in read_request_line (bb=<optimized out>, r=<optimized out>) at protocol.c:590
        rv = 20674
        ll = 0x5 <error: Cannot access memory at address 0x5>
        pro = 0x64 <error: Cannot access memory at address 0x64>
        major = 1
        minor = 0
        http = "\240\060\n\360?"
        len = 139912586867384
        num_blank_lines = 0
        max_blank_lines = 100
        uri = 0x64 <error: Cannot access memory at address 0x64>
#18 ap_read_request (conn=0x7f3ff00a32b8) at protocol.c:960
        p = 0x7f3ff009c028
        expect = 0x64 <error: Cannot access memory at address 0x64>
        access_status = -335550640
        tmp_bb = 0x7f3ff009d8b0
        csd = 0x64
        cur_timeout = 139912586867384
#19 0x00007f3ffdebed5e in ap_process_http_sync_connection (c=0x7f3ff00a32b8) at http_core.c:181
        r = <optimized out>
        cs = 0x0
        csd = 0x0
        mpm_state = 0
#20 ap_process_http_connection (c=0x7f3ff00a32b8) at http_core.c:231
No locals.
#21 0x00007f3ffdeb5790 in ap_run_process_connection (c=0x7f3ff00a32b8) at connection.c:41
        pHook = <optimized out>
        n = 2
        rv = -1
#22 0x00007f3ffa0b851b in process_socket (bucket_alloc=<optimized out>, my_thread_num=<optimized out>, my_child_num=<optimized out>, sock=<optimized out>, p=<optimized out>, thd=<optimized out>) at worker.c:619
        current_conn = 0x7f3ff00a32b8
        conn_id = 139912586867384
        sbh = 0x7f3ff00a32b0
#23 worker_thread (thd=0x7f3febffe750, dummy=0x1) at worker.c:978
        process_slot = 0
        thread_slot = 6
        csd = 0x7f3ff00a30a0
        bucket_alloc = 0x0
        last_ptrans = 0x7f3ff00a30a0
        ptrans = 0x7f3ff00a3028
        is_idle = -267767112
#24 0x00007f3ffd3500a4 in start_thread (arg=0x7f3febfff700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3febfff700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912519087872, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912519087872, 3304068413592615258, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#25 0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 24 (Thread 0x7f3feb7fe700 (LWP 41767)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3feb7fde80, p=0x7f3feb7fde88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 7
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3feb7fe700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3feb7fe700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912510695168, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912510695168, 3304069511493630298, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 23 (Thread 0x7f3feaffd700 (LWP 41768)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3feaffce80, p=0x7f3feaffce88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 8
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3feaffd700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3feaffd700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912502302464, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912502302464, 3304070611542128986, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 22 (Thread 0x7f3fea7fc700 (LWP 41769)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fea7fbe80, p=0x7f3fea7fbe88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 9
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fea7fc700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fea7fc700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912493909760, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912493909760, 3304071713738111322, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 21 (Thread 0x7f3fe9ffb700 (LWP 41770)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe9ffae80, p=0x7f3fe9ffae88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 10
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe9ffb700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe9ffb700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912485517056, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912485517056, 3304072813786610010, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 20 (Thread 0x7f3fe97fa700 (LWP 41771)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe97f9e80, p=0x7f3fe97f9e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 11
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe97fa700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe97fa700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912477124352, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912477124352, 3304073911687625050, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 19 (Thread 0x7f3fe8ff9700 (LWP 41772)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe8ff8e80, p=0x7f3fe8ff8e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 12
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe8ff9700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe8ff9700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912468731648, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912468731648, 3304075011736123738, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 18 (Thread 0x7f3fe87f8700 (LWP 41773)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe87f7e80, p=0x7f3fe87f7e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 13
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe87f8700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe87f8700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912460338944, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912460338944, 3304076105342171482, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 17 (Thread 0x7f3fe27ec700 (LWP 41785)):
#0  0x00007f3ffd085623 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
No locals.
#1  0x00007f3ffd587fc3 in impl_pollset_poll (pollset=0x7f3ffdc720a0, timeout=<optimized out>, num=0x7f3fe27ebe54, descriptors=0x7f3fe27ebe70) at /tmp/buildd/apr-1.5.1/poll/unix/epoll.c:256
        ret = <optimized out>
        i = <optimized out>
        j = <optimized out>
        rv = 0
        fdptr = <optimized out>
#2  0x00007f3ffa0b8fcd in listener_thread (thd=0x10, dummy=0x7f3ffe0f4818 <ap_listeners>) at worker.c:779
        numdesc = 1
        pdesc = 0x7f3fec0031e0
        tpool = 0x7f3fe27ebe70
        csd = 0x7f3ff00a30a0
        ptrans = 0x0
        pollset = 0x7f3ffdc720a0
        have_idle_worker = 2
        last_poll_idx = 1
#3  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe27ec700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe27ec700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912359626496, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912359626496, 3304089297334221146, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#4  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 16 (Thread 0x7f3fe7ff7700 (LWP 41774)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe7ff6e80, p=0x7f3fe7ff6e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 14
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe7ff7700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe7ff7700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912451946240, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912451946240, 3304077205390670170, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 15 (Thread 0x7f3ff30d3700 (LWP 41760)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3ff30d2e80, p=0x7f3ff30d2e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 0
        csd = 0x7f3ffdc700a0
        bucket_alloc = 0x2
        last_ptrans = 0x0
        ptrans = 0x7f3ffdc70028
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3ff30d3700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3ff30d3700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912637396736, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912637396736, 3304051489273985370, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 14 (Thread 0x7f3fe77f6700 (LWP 41775)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe77f5e80, p=0x7f3fe77f5e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 15
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe77f6700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe77f6700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912443553536, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912443553536, 3304078303291685210, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 13 (Thread 0x7f3ff28d2700 (LWP 41761)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3ff28d1e80, p=0x7f3ff28d1e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 1
        csd = 0x7f3ffdc520a0
        bucket_alloc = 0x2
        last_ptrans = 0x0
        ptrans = 0x7f3ffdc52028
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3ff28d2700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3ff28d2700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912629004032, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912629004032, 3304052587175000410, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 12 (Thread 0x7f3fe6ff5700 (LWP 41776)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe6ff4e80, p=0x7f3fe6ff4e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 16
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe6ff5700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe6ff5700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912435160832, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912435160832, 3304079403340183898, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 11 (Thread 0x7f3fe67f4700 (LWP 41777)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe67f3e80, p=0x7f3fe67f3e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 17
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe67f4700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe67f4700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912426768128, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912426768128, 3304080505536166234, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 10 (Thread 0x7f3ff10cf700 (LWP 41764)):
#0  0x00007f3ffd07c50d in poll () at ../sysdeps/unix/syscall-template.S:81
No locals.
#1  0x00007f3ffd58b4db in poll (__timeout=5000, __nfds=1, __fds=0x7f3ff10ce750) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
No locals.
#2  apr_wait_for_io_or_timeout (f=f@entry=0x0, s=s@entry=0x7f3ffdc520b0, for_read=for_read@entry=1) at /tmp/buildd/apr-1.5.1/support/unix/waitio.c:51
        pfd = {fd = 20, events = 1, revents = 0}
        timeout = <optimized out>
#3  0x00007f3ffd58523a in apr_socket_recv (sock=sock@entry=0x7f3ffdc520b0, 
    buf=buf@entry=0x7f3ff0083048 "]\247\231\314\r$B\022c\212\340\200\240\024\067\347\177\326o\375c]\274\220\004\337dÑ?\247\271\213\070\036Y\004v\201\305b\261\317\002X=8\t]4wq\020\364kS<C\303\071w\250\027\324\006\017\020<'\256\023U\326\006'\030\255\030Y\364k\017kȱ\373\265\021\016V3\212\231\335\351v;GSg~I\370\363`\216],\232\070\243\360n×?\215\071>Z8.'\254_Ó?\203W\304\361\bZ\222\300'\214q\342!\305\035\240z\247q\322Æ?\223\266\353\334Y\202WW?\205$$AF\206\270x\020u\352xh-\004\362\220\252A\365\343\317)Å?\260\215\063/\220#\256\060%y\245$\211\352vjdB\\"..., len=len@entry=0x7f3ff10ce810) at /tmp/buildd/apr-1.5.1/network_io/unix/sendrecv.c:87
        rv = <optimized out>
        arv = <optimized out>
#4  0x00007f3ffd7a5fc1 in socket_bucket_read (a=0x7f3ff00cc108, str=0x7f3ff10ce808, len=0x7f3ff10ce810, block=<optimized out>) at /tmp/buildd/apr-util-1.5.4/buckets/apr_buckets_socket.c:36
        p = 0x7f3ffdc520b0
        buf = 0x7f3ff0083048 "]\247\231\314\r$B\022c\212\340\200\240\024\067\347\177\326o\375c]\274\220\004\337dÑ?\247\271\213\070\036Y\004v\201\305b\261\317\002X=8\t]4wq\020\364kS<C\303\071w\250\027\324\006\017\020<'\256\023U\326\006'\030\255\030Y\364k\017kȱ\373\265\021\016V3\212\231\335\351v;GSg~I\370\363`\216],\232\070\243\360n×?\215\071>Z8.'\254_Ó?\203W\304\361\bZ\222\300'\214q\342!\305\035\240z\247q\322Æ?\223\266\353\334Y\202WW?\205$$AF\206\270x\020u\352xh-\004\362\220\252A\365\343\317)Å?\260\215\063/\220#\256\060%y\245$\211\352vjdB\\"...
        rv = <optimized out>
        timeout = 139912970272384
#5  0x00007f3ffdea6fc1 in ap_core_input_filter (f=0x7f3ff00b6120, b=0x7f3ff00b60a0, mode=<optimized out>, block=APR_BLOCK_READ, readbytes=5) at core_filters.c:236
        e = 0x7f3ff00cc108
        rv = <optimized out>
        net = <optimized out>
        ctx = 0x7f3ff00b6278
        str = 0x0
        len = 8000
#6  0x00007f3ffdecb1ba in logio_in_filter (f=<optimized out>, bb=0x7f3ff00b60a0, mode=<optimized out>, block=<optimized out>, readbytes=<optimized out>) at mod_logio.c:140
        length = 1430824257
        status = <optimized out>
        cf = 0x7f3ffdc52688
#7  0x00007f3ff93a8a1a in reqtimeout_filter (f=0x7f3ff10ce750, bb=0x7f3ff00b60a0, mode=AP_MODE_READBYTES, block=APR_BLOCK_READ, readbytes=5) at mod_reqtimeout.c:303
        now = 20000000
        saved_sock_timeout = 5000000
        ccfg = 0x7f3ff00b6170
#8  0x00007f3ff8d7a2ea in bio_filter_in_read (bio=0x7f3fd0001650, in=0x7f3fd0006f83 "\320?\177", inlen=5) at ssl_engine_io.c:471
        inl = 5
        inctx = 0x7f3ff00b4028
        block = APR_BLOCK_READ
#9  0x00007f3ff87fe16c in BIO_read () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
No symbol table info available.
#10 0x00007f3ff8b283a4 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#11 0x00007f3ff8b2967d in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#12 0x00007f3ff8b26994 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#13 0x00007f3ff8d791ce in ssl_io_input_read (inctx=0x7f3ff00b4028, 
    buf=0x7f3ff00b4070 "Action=AgentDashboard;Subaction=Element;Name=0110-TicketEscalation;Filter=All;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQl, */*; q=0.01\r\nAccept-Language: de,en-US;q=0."..., 
    len=0x7f3ff10cec60) at ssl_engine_io.c:591
        wanted = 8192
        bytes = 139912586936432
        rc = -516
#14 0x00007f3ff8d7b8eb in ssl_io_input_getline (len=0x7f3ff10cec58, 
    buf=0x7f3ff00b4070 "Action=AgentDashboard;Subaction=Element;Name=0110-TicketEscalation;Filter=All;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQl, */*; q=0.01\r\nAccept-Language: de,en-US;q=0."..., 
    inctx=0x7f3ff00b4028) at ssl_engine_io.c:709
        status = <optimized out>
        tmplen = 0
        buflen = 8192
        pos = 0x0
        offset = 0
#15 ssl_io_filter_input (f=0x7f3ff00b6078, bb=0x7f3ff00b38b0, mode=<optimized out>, block=<optimized out>, readbytes=<optimized out>) at ssl_engine_io.c:1388
        pos = <optimized out>
        status = 0
        inctx = 0x7f3ff00b4028
        start = 0x7f3ff00b4070 "Action=AgentDashboard;Subaction=Element;Name=0110-TicketEscalation;Filter=All;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQl, */*; q=0.01\r\nAccept-Language: de,en-US;q=0."...
        len = 0
        is_init = <optimized out>
#16 0x00007f3ffde95e83 in ap_rgetline_core (s=0x7f3ff00b20d0, n=8192, read=0x7f3ff10ced98, r=0x7f3ff00b20a0, fold=0, bb=0x7f3ff00b38b0) at protocol.c:229
        rv = <optimized out>
        e = <optimized out>
        bytes_handled = 0
        current_alloc = 0
        pos = <optimized out>
        last_char = 0x0
        do_alloc = <optimized out>
        saw_eos = 0
#17 0x00007f3ffde9818e in read_request_line (bb=<optimized out>, r=<optimized out>) at protocol.c:590
        rv = 5000
        ll = 0x0
        pro = 0x64 <error: Cannot access memory at address 0x64>
        major = 1
        minor = 0
        http = "\310\"\305\375?"
        len = 139912819580414
        num_blank_lines = 0
        max_blank_lines = 100
        uri = 0x64 <error: Cannot access memory at address 0x64>
#18 ap_read_request (conn=0x7f3ffdc522c8) at protocol.c:960
        p = 0x7f3ff00b2028
        expect = 0x64 <error: Cannot access memory at address 0x64>
        access_status = -250812592
        tmp_bb = 0x7f3ff00b38b0
        csd = 0x64
        cur_timeout = 139912819580414
#19 0x00007f3ffdebed5e in ap_process_http_sync_connection (c=0x7f3ffdc522c8) at http_core.c:181
        r = <optimized out>
        cs = 0x0
        csd = 0x7f3ffdc520b0
        mpm_state = 1
#20 ap_process_http_connection (c=0x7f3ffdc522c8) at http_core.c:231
No locals.
#21 0x00007f3ffdeb5790 in ap_run_process_connection (c=0x7f3ffdc522c8) at connection.c:41
        pHook = <optimized out>
        n = 2
        rv = -1
#22 0x00007f3ffa0b851b in process_socket (bucket_alloc=<optimized out>, my_thread_num=<optimized out>, my_child_num=<optimized out>, sock=<optimized out>, p=<optimized out>, thd=<optimized out>) at worker.c:619
        current_conn = 0x7f3ffdc522c8
        conn_id = 139912817222344
        sbh = 0x7f3ffdc522c0
#23 worker_thread (thd=0x7f3ff10ce750, dummy=0x1) at worker.c:978
        process_slot = 0
        thread_slot = 4
        csd = 0x7f3ffdc520b0
        bucket_alloc = 0x0
        last_ptrans = 0x7f3ffdc520b0
        ptrans = 0x7f3ffdc52028
        is_idle = -37412152
#24 0x00007f3ffd3500a4 in start_thread (arg=0x7f3ff10cf700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3ff10cf700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912603825920, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912603825920, 3304055880878045530, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#25 0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 9 (Thread 0x7f3fe5ff3700 (LWP 41778)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe5ff2e80, p=0x7f3fe5ff2e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 18
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe5ff3700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe5ff3700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912418375424, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912418375424, 3304081605584664922, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 8 (Thread 0x7f3fe57f2700 (LWP 41779)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe57f1e80, p=0x7f3fe57f1e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 19
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe57f2700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe57f2700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912409982720, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912409982720, 3304082703485679962, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 7 (Thread 0x7f3fe4ff1700 (LWP 41780)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe4ff0e80, p=0x7f3fe4ff0e88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 20
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe4ff1700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe4ff1700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912401590016, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912401590016, 3304083803534178650, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 6 (Thread 0x7f3fe47f0700 (LWP 41781)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe47efe80, p=0x7f3fe47efe88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 21
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe47f0700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe47f0700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912393197312, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912393197312, 3304084897140226394, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 5 (Thread 0x7f3fe3fef700 (LWP 41782)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe3feee80, p=0x7f3fe3feee88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 22
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe3fef700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe3fef700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912384804608, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912384804608, 3304085997188725082, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 4 (Thread 0x7f3fe37ee700 (LWP 41783)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe37ede80, p=0x7f3fe37ede88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 23
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe37ee700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe37ee700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912376411904, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912376411904, 3304087095089740122, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 3 (Thread 0x7f3fe2fed700 (LWP 41784)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
No locals.
#1  0x00007f3ffd58077d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at /tmp/buildd/apr-1.5.1/locks/unix/thread_cond.c:68
No locals.
#2  0x00007f3ffa0baaed in ap_queue_pop (queue=0x7f3ffddb81c0, sd=0x7f3fe2fece80, p=0x7f3fe2fece88) at fdqueue.c:352
        elem = <optimized out>
        rv = <optimized out>
#3  0x00007f3ffa0b83bf in worker_thread (thd=0x7f3ffddb824c, dummy=0x80) at worker.c:945
        process_slot = 0
        thread_slot = 24
        csd = 0x0
        bucket_alloc = 0x0
        last_ptrans = 0x0
        ptrans = 0x0
        is_idle = 0
#4  0x00007f3ffd3500a4 in start_thread (arg=0x7f3fe2fed700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3fe2fed700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912368019200, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912368019200, 3304088195138238810, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#5  0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 2 (Thread 0x7f3ff18d0700 (LWP 41763)):
#0  0x00007f3ffd07c50d in poll () at ../sysdeps/unix/syscall-template.S:81
No locals.
#1  0x00007f3ffd58b4db in poll (__timeout=5000, __nfds=1, __fds=0x7f3ff18cf750) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
No locals.
#2  apr_wait_for_io_or_timeout (f=f@entry=0x0, s=s@entry=0x7f3ffdc700b0, for_read=for_read@entry=1) at /tmp/buildd/apr-1.5.1/support/unix/waitio.c:51
        pfd = {fd = 19, events = 1, revents = 0}
        timeout = <optimized out>
#3  0x00007f3ffd58523a in apr_socket_recv (sock=sock@entry=0x7f3ffdc700b0, 
    buf=buf@entry=0x7f3ffdc65048 ".\223\351@\"$1\246H\r\b\332q\257~\233>F\377Ë?\335\002\340\a\bR\022%\227\275:&\333\310\024\200\305.\026\213\337.Ip\365\350\300\267\321\310Æ?ש\234\343!:\034\061\313u\274J\367\021\202\343\300\266\302\312", <incomplete sequence \367>, len=len@entry=0x7f3ff18cf810) at /tmp/buildd/apr-1.5.1/network_io/unix/sendrecv.c:87
        rv = <optimized out>
        arv = <optimized out>
#4  0x00007f3ffd7a5fc1 in socket_bucket_read (a=0x7f3ffdc6e388, str=0x7f3ff18cf808, len=0x7f3ff18cf810, block=<optimized out>) at /tmp/buildd/apr-util-1.5.4/buckets/apr_buckets_socket.c:36
        p = 0x7f3ffdc700b0
        buf = 0x7f3ffdc65048 ".\223\351@\"$1\246H\r\b\332q\257~\233>F\377Ë?\335\002\340\a\bR\022%\227\275:&\333\310\024\200\305.\026\213\337.Ip\365\350\300\267\321\310Æ?ש\234\343!:\034\061\313u\274J\367\021\202\343\300\266\302\312", <incomplete sequence \367>
        rv = <optimized out>
        timeout = 139912942678720
#5  0x00007f3ffdea6fc1 in ap_core_input_filter (f=0x7f3ffdc6d120, b=0x7f3ffdc6d0a0, mode=<optimized out>, block=APR_BLOCK_READ, readbytes=5) at core_filters.c:236
        e = 0x7f3ffdc6e388
        rv = <optimized out>
        net = <optimized out>
        ctx = 0x7f3ffdc6d278
        str = 0x0
        len = 8000
#6  0x00007f3ffdecb1ba in logio_in_filter (f=<optimized out>, bb=0x7f3ffdc6d0a0, mode=<optimized out>, block=<optimized out>, readbytes=<optimized out>) at mod_logio.c:140
        length = 1430824257
        status = <optimized out>
        cf = 0x7f3ffdc70688
#7  0x00007f3ff93a8a1a in reqtimeout_filter (f=0x7f3ff18cf750, bb=0x7f3ffdc6d0a0, mode=AP_MODE_READBYTES, block=APR_BLOCK_READ, readbytes=5) at mod_reqtimeout.c:303
        now = 20000000
        saved_sock_timeout = 5000000
        ccfg = 0x7f3ffdc6d170
#8  0x00007f3ff8d7a2ea in bio_filter_in_read (bio=0x7f3fcc001650, in=0x7f3fdc006f83 "\320?\177", inlen=5) at ssl_engine_io.c:471
        inl = 5
        inctx = 0x7f3ffdc6b028
        block = APR_BLOCK_READ
#9  0x00007f3ff87fe16c in BIO_read () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
No symbol table info available.
#10 0x00007f3ff8b283a4 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#11 0x00007f3ff8b2967d in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#12 0x00007f3ff8b26994 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#13 0x00007f3ff8d791ce in ssl_io_input_read (inctx=0x7f3ffdc6b028, 
    buf=0x7f3ffdc6b070 "Action=AgentDashboard;Subaction=Element;Name=0130-TicketOpen;Filter=MyQueues;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQml, */*; q=0.01\r\nAccept-Language: de,en-US;q=0."..., 
    len=0x7f3ff18cfc60) at ssl_engine_io.c:591
        wanted = 8192
        bytes = 139912817324144
        rc = -516
#14 0x00007f3ff8d7b8eb in ssl_io_input_getline (len=0x7f3ff18cfc58, 
    buf=0x7f3ffdc6b070 "Action=AgentDashboard;Subaction=Element;Name=0130-TicketOpen;Filter=MyQueues;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQml, */*; q=0.01\r\nAccept-Language: de,en-US;q=0."..., 
    inctx=0x7f3ffdc6b028) at ssl_engine_io.c:709
        status = <optimized out>
        tmplen = 0
        buflen = 8192
        pos = 0x0
        offset = 0
#15 ssl_io_filter_input (f=0x7f3ffdc6d078, bb=0x7f3ffdc648b0, mode=<optimized out>, block=<optimized out>, readbytes=<optimized out>) at ssl_engine_io.c:1388
        pos = <optimized out>
        status = 0
        inctx = 0x7f3ffdc6b028
        start = 0x7f3ffdc6b070 "Action=AgentDashboard;Subaction=Element;Name=0130-TicketOpen;Filter=MyQueues;CustomerID=;SortBy=;OrderBy=;ChallengeToken=8JSMnp2DIamgjV9ne1wrcKjivWRTzWrQml, */*; q=0.01\r\nAccept-Language: de,en-US;q=0."...
        len = 0
        is_init = <optimized out>
#16 0x00007f3ffde95e83 in ap_rgetline_core (s=0x7f3ffdc630d0, n=8192, read=0x7f3ff18cfd98, r=0x7f3ffdc630a0, fold=0, bb=0x7f3ffdc648b0) at protocol.c:229
        rv = <optimized out>
        e = <optimized out>
        bytes_handled = 0
        current_alloc = 0
        pos = <optimized out>
        last_char = 0x0
        do_alloc = <optimized out>
        saw_eos = 0
#17 0x00007f3ffde9818e in read_request_line (bb=<optimized out>, r=<optimized out>) at protocol.c:590
        rv = 5000
        ll = 0x0
        pro = 0x64 <error: Cannot access memory at address 0x64>
        major = 1
        minor = 0
        http = "\310\002\307\375?"
        len = 139912819580414
        num_blank_lines = 0
        max_blank_lines = 100
        uri = 0x64 <error: Cannot access memory at address 0x64>
#18 ap_read_request (conn=0x7f3ffdc702c8) at protocol.c:960
        p = 0x7f3ffdc63028
        expect = 0x64 <error: Cannot access memory at address 0x64>
        access_status = -242419888
        tmp_bb = 0x7f3ffdc648b0
        csd = 0x64
        cur_timeout = 139912819580414
#19 0x00007f3ffdebed5e in ap_process_http_sync_connection (c=0x7f3ffdc702c8) at http_core.c:181
        r = <optimized out>
        cs = 0x0
        csd = 0x7f3ffdc700b0
        mpm_state = 1
#20 ap_process_http_connection (c=0x7f3ffdc702c8) at http_core.c:231
No locals.
#21 0x00007f3ffdeb5790 in ap_run_process_connection (c=0x7f3ffdc702c8) at connection.c:41
        pHook = <optimized out>
        n = 2
        rv = -1
#22 0x00007f3ffa0b851b in process_socket (bucket_alloc=<optimized out>, my_thread_num=<optimized out>, my_child_num=<optimized out>, sock=<optimized out>, p=<optimized out>, thd=<optimized out>) at worker.c:619
        current_conn = 0x7f3ffdc702c8
        conn_id = 139912817345224
        sbh = 0x7f3ffdc702c0
#23 worker_thread (thd=0x7f3ff18cf750, dummy=0x1) at worker.c:978
        process_slot = 0
        thread_slot = 3
        csd = 0x7f3ffdc700b0
        bucket_alloc = 0x0
        last_ptrans = 0x7f3ffdc700b0
        ptrans = 0x7f3ffdc70028
        is_idle = -37289272
#24 0x00007f3ffd3500a4 in start_thread (arg=0x7f3ff18d0700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3ff18d0700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912612218624, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912612218624, 3304054780829546842, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#25 0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Thread 1 (Thread 0x7f3ff08ce700 (LWP 41765)):
#0  0x00007f3ff9c7dac2 in modperl_interp_unselect () from /usr/lib/apache2/modules/mod_perl.so
No symbol table info available.
#1  0x00007f3ffd5819be in run_cleanups (cref=<optimized out>) at /tmp/buildd/apr-1.5.1/memory/unix/apr_pools.c:2352
        c = <optimized out>
#2  apr_pool_destroy (pool=0x7f3ff00a9028) at /tmp/buildd/apr-1.5.1/memory/unix/apr_pools.c:814
        active = <optimized out>
        allocator = <optimized out>
#3  0x00007f3ff8d7bc24 in ssl_io_filter_output (f=0x7f3ff00b0830, bb=0x7f3ff00adb58) at ssl_engine_io.c:1659
        data = 0x0
        len = 0
        bucket = 0x7f3ff00ae1a8
        filter_ctx = 0x7f3ff00b07d8
        inctx = 0x0
        rblock = APR_BLOCK_READ
#4  0x00007f3ff8d7877a in ssl_io_filter_coalesce (f=0x0, bb=0x7f3ff00adb58) at ssl_engine_io.c:1558
        ctx = 0x7f3ff00ad350
        count = 0
#5  0x00007f3ffdec17bd in ap_process_request_after_handler (r=0x7f3ff00a90a0) at http_request.c:256
        bb = 0x7f3ff00adb58
        b = <optimized out>
        c = 0x7f3ff00b02b8
#6  0x00007f3ffdec24a0 in ap_process_request (r=0x7f3ff00a90a0) at http_request.c:363
        bb = 0x0
        c = 0x7f3ff00b02b8
        rv = 0
#7  0x00007f3ffdebeda2 in ap_process_http_sync_connection (c=0x7f3ff00b02b8) at http_core.c:190
        r = 0x7f3ff00a90a0
        cs = 0x0
        csd = 0x0
        mpm_state = 0
#8  ap_process_http_connection (c=0x7f3ff00b02b8) at http_core.c:231
No locals.
#9  0x00007f3ffdeb5790 in ap_run_process_connection (c=0x7f3ff00b02b8) at connection.c:41
        pHook = <optimized out>
        n = 2
        rv = -1
#10 0x00007f3ffa0b851b in process_socket (bucket_alloc=<optimized out>, my_thread_num=<optimized out>, my_child_num=<optimized out>, sock=<optimized out>, p=<optimized out>, thd=<optimized out>) at worker.c:619
        current_conn = 0x7f3ff00b02b8
        conn_id = 139912586920632
        sbh = 0x7f3ff00b02b0
#11 worker_thread (thd=0x0, dummy=0x0) at worker.c:978
        process_slot = 0
        thread_slot = 5
        csd = 0x7f3ff00b00a0
        bucket_alloc = 0x0
        last_ptrans = 0x7f3ff00b00a0
        ptrans = 0x7f3ff00b0028
        is_idle = -267713864
#12 0x00007f3ffd3500a4 in start_thread (arg=0x7f3ff08ce700) at pthread_create.c:309
        __res = <optimized out>
        pd = 0x7f3ff08ce700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139912595433216, -3195938041570608806, 0, 139912819318880, 139912822078584, 139912595433216, 3304056978779060570, 3304029291995248986}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
              0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#13 0x00007f3ffd08504d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

Reply to: