Bug#814980: apache2 using authnz_ldap: Infinite loop in find_block_of_size of authn_ldap_check_password - hanging processes hang whole server eventually
Package: apache2
Version: 2.4.10-10+deb8u4
Severity: important
Dear Maintainer,
after upgrading from wheezy to jessie and adjusting our config we
experienced hangups in the apache2 processes. Attaching to the running
processes with gdb gives the following backtrace:
(gdb) bt
#0 0x00007f2a4aa4b5ad in find_block_of_size (size=size@entry=48,
rmm=0x7f2a4b2d5148) at /tmp/buildd/apr-util-1.5.4/misc/apr_rmm.c:106
#1 0x00007f2a4aa4bdd8 in apr_rmm_calloc (rmm=0x7f2a4b2d5148,
reqsize=<optimized out>) at
/tmp/buildd/apr-util-1.5.4/misc/apr_rmm.c:342
#2 0x00007f2a43c31fad in util_ald_alloc (cache=0x7f2a3e9c1c88,
size=<optimized out>) at util_ldap_cache_mgr.c:105
#3 0x00007f2a43c3277b in util_ald_cache_insert (cache=0x7f2a3e9c1008,
payload=0x30) at util_ldap_cache_mgr.c:470
#4 0x00007f2a43c2fa86 in uldap_cache_checkuserid (r=0x0,
ldc=0x7f2a4b1c10a0, url=0x7ffefb9c6930 "xxxxxxxxxx",
basedn=0x7f2a3e9ccf90 "", scope=160, attrs=0x3b90,
filter=0x7ffefb9c6a60
"(&(objectClass=user)(sAMAccountName=xxxxxxxx))",
bindpw=0x7f2a4aeee890 "xxxxxxxxxx", binddn=0x7ffefb9c69f8,
retvals=0x7f2a4aeee8b8) at util_ldap.c:1880
#5 0x00007f2a48ba9a1d in authn_ldap_check_password
(r=0x7f2a1ec790a0, user=0x30 <error: Cannot access memory at address
0x30>, password=0x7f2a4aeee890 "xxxxxxxxxxx") at
mod_authnz_ldap.c:543
#6 0x00007f2a491b8a76 in authenticate_basic_user (r=0x7f2a1ec790a0)
at mod_auth_basic.c:383
#7 0x00007f2a4b139070 in ap_run_check_user_id (r=0x7f2a1ec790a0) at
request.c:81
#8 0x00007f2a4b13c5b4 in ap_process_request_internal
(r=0x7f2a1ec790a0) at request.c:273
#9 0x00007f2a4b159670 in ap_process_async_request
(r=0x7f2a1ec790a0) at http_request.c:315
#10 0x00007f2a4b159820 in ap_process_request (r=0x7f2a1ec790a0) at
http_request.c:363
#11 0x00007f2a4b156122 in ap_process_http_sync_connection
(c=0x7f2a4aeff290) at http_core.c:190
#12 ap_process_http_connection (c=0x7f2a4aeff290) at http_core.c:231
#13 0x00007f2a4b14cb10 in ap_run_process_connection
(c=0x7f2a4aeff290) at connection.c:41
#14 0x00007f2a4381b7ba in child_main (child_num_arg=1050415112) at
prefork.c:704
#15 0x00007f2a4381ba01 in make_child (s=0x7f2a4b34ade0, slot=16) at
prefork.c:800
#16 0x00007f2a4381c667 in perform_idle_server_maintenance
(p=<optimized out>) at prefork.c:902
#17 prefork_run (_pconf=0x7f2a4b389f38 <ap_server_conf>,
plog=0x7ffefb9c8d5c, s=0x7ffefb9c8d60) at prefork.c:1090
#18 0x00007f2a4b128e7e in ap_run_mpm (pconf=0x7f2a4b378028,
plog=0x7f2a4b346028, s=0x7f2a4b34ade0) at mpm_common.c:94
#19 0x00007f2a4b1223c3 in main (argc=3, argv=0x7ffefb9c9048) at
main.c:777
(gdb)
We suspect that the heap is corrupted in the shared memory segment.
All apache child processes start getting affected by this at the same
time which indicates further that the shared memory is the problem.
Thanks for looking into this.
Rainer
-- Package-specific info:
-- System Information:
Debian Release: 8.3
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages apache2 depends on:
ii apache2-bin 2.4.10-10+deb8u4
ii apache2-data 2.4.10-10+deb8u4
ii apache2-utils 2.4.10-10+deb8u4
ii dpkg 1.17.26
ii lsb-base 4.1+Debian13+nmu1
ii mime-support 3.58
ii perl 5.20.2-3+deb8u3
ii procps 2:3.3.9-9
Versions of packages apache2 recommends:
ii ssl-cert 1.0.35
Versions of packages apache2 suggests:
pn apache2-doc <none>
pn apache2-suexec-pristine | apache2-suexec-custom <none>
ii w3m [www-browser] 0.5.3-19
Versions of packages apache2-bin depends on:
ii libapr1 1.5.1-3
ii libaprutil1 1.5.4-1
ii libaprutil1-dbd-sqlite3 1.5.4-1
ii libaprutil1-ldap 1.5.4-1
ii libc6 2.19-18+deb8u2
ii libldap-2.4-2 2.4.40+dfsg-1+deb8u2
ii liblua5.1-0 5.1.5-7.1
ii libpcre3 2:8.35-3.3+deb8u2
ii libssl1.0.0 1.0.1k-3+deb8u2
ii libxml2 2.9.1+dfsg1-5+deb8u1
ii perl 5.20.2-3+deb8u3
ii zlib1g 1:1.2.8.dfsg-2+b1
Versions of packages apache2-bin suggests:
pn apache2-doc <none>
pn apache2-suexec-pristine | apache2-suexec-custom <none>
ii w3m [www-browser] 0.5.3-19
Versions of packages apache2 is related to:
ii apache2 2.4.10-10+deb8u4
ii apache2-bin 2.4.10-10+deb8u4
-- Configuration Files:
/etc/apache2/apache2.conf changed:
Mutex file:${APACHE_LOCK_DIR} default
PidFile ${APACHE_PID_FILE}
Timeout 300
KeepAlive Off
MaxKeepAliveRequests 100
KeepAliveTimeout 5
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}
HostnameLookups Off
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel warn
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf
Include ports.conf
<Directory />
Options FollowSymLinks
AllowOverride None
Require all denied
</Directory>
<Directory /usr/share>
AllowOverride None
Require all granted
</Directory>
<Directory /var/www/>
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
AccessFileName .htaccess
<FilesMatch "^\.ht">
Require all denied
</FilesMatch>
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%{X-Forwarded-For}i %h %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %h %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %D" combined_reqtime
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
IncludeOptional conf-enabled/*.conf
IncludeOptional sites-enabled/*.conf
/etc/apache2/conf-available/security.conf changed:
ServerTokens Prod
ServerSignature Off
TraceEnable Off
<Directory /home>
Options FollowSymLinks
AllowOverride All
Require all granted
</Directory>
<DirectoryMatch "/home/wwwdocs/public_html/video.godmode-trader.de/">
RewriteEngine off
Options None
AllowOverride None
php_admin_flag engine off
<FilesMatch "\.phps?$">
SetHandler default-handler
Require all denied
</FilesMatch>
</DirectoryMatch>
<DirectoryMatch "/\.(svn|git)">
Require all denied
</DirectoryMatch>
/etc/apache2/envvars changed:
unset HOME
if [ "${APACHE_CONFDIR##/etc/apache2-}" != "${APACHE_CONFDIR}" ] ; then
SUFFIX="-${APACHE_CONFDIR##/etc/apache2-}"
else
SUFFIX=
fi
export APACHE_RUN_USER=wwwdocs
export APACHE_RUN_GROUP=wwwdocs
export APACHE_PID_FILE=/var/run/apache2/apache2$SUFFIX.pid
export APACHE_RUN_DIR=/var/run/apache2$SUFFIX
export APACHE_LOCK_DIR=/var/lock/apache2$SUFFIX
export APACHE_LOG_DIR=/var/log/apache2$SUFFIX
export LANG=C
export LANG
umask 002
/etc/apache2/mods-available/deflate.conf changed:
<IfModule mod_filter.c>
# these are known to be safe with MSIE 6
AddOutputFilterByType DEFLATE text/html text/plain text/xml
# everything else may cause problems with MSIE 6
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript text/javascript
AddOutputFilterByType DEFLATE application/atom+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/json
AddOutputFilterByType DEFLATE image/svg+xml
</IfModule>
/etc/apache2/mods-available/dir.conf changed:
<IfModule mod_dir.c>
DirectoryIndex index.php index.html
</IfModule>
/etc/apache2/mods-available/mpm_prefork.conf changed:
<IfModule mpm_prefork_module>
StartServers 32
MinSpareServers 15
MaxSpareServers 25
MaxRequestWorkers 128
MaxConnectionsPerChild 0
</IfModule>
/etc/apache2/mods-available/status.conf changed:
<IfModule mod_status.c>
<Location /server-status>
SetHandler server-status
Require local
Require ip 10.20.35.0/24
Require ip 10.20.50.0/24
Require ip 10.20.56.0/24
Require ip 192.168.0.0/22
</Location>
# Keep track of extended status information for each request
ExtendedStatus On
# Determine if mod_status displays the first 63 characters of a request or
# the last 63, assuming the request itself is greater than 63 chars.
# Default: Off
#SeeRequestTail On
<IfModule mod_proxy.c>
# Show Proxy LoadBalancer status in mod_status
ProxyStatus On
</IfModule>
</IfModule>
/etc/apache2/ports.conf changed:
Listen 80
<IfModule ssl_module>
Listen 443
</IfModule>
<IfModule mod_gnutls.c>
Listen 443
</IfModule>
-- no debconf information
Reply to: