Bug#934080: [libc6] Significant degradation in the memory effectivity of the memory allocator
Hi,
On 2019-08-06 23:01, Roman Savochenko wrote:
> Package: libc6
> Version: 2.19, 2.24, 2.28
> Severity: normal
>
> --- Please enter the report below this line. ---
> Initial condition of the problem representing is a program in the single
> source code, built on-and for Debian 7, 8, 9, 10 with a result in the Live
> disks.
>
> The program represents a web-interface of several pages, where here used
> only the first page.
>
> In building the first page there used wide range of memory chunks: small
> objects of the C++ classes (~100 bytes), resources of the image files (~10
> kbytes), GD memory blocks (~1 kbytes), so on
>
> The live disks were started under VirtualBox 5.2, where the got data was
> measured by *top*.
Can you details more precisely on you measure the memory used? Do you
just get the line corresponding to the process you want to monitor?
Which column do you take?
> The data measuring under VirtualBox performed into the next stages:
> 1. Start the program and set the initial state, fixing the memory allocation
> — *measuring the initial memory consumption value*
> 2. Perform the allocation-freeing iteration
> 2.1. Open the first Web-interface page from a Web-browser of the host system
> 2.2. Close the page on the Web-browser
> 2.3. Wait to close-freeing session of the first Web-interface page on the
> program side, 1 minute — *measuring the iteration **memory consumption
> value*
> 3. Return to the stage 2 and repeating 5 iterations
>
> The stage 2.3 tested to real freeing all the allocated memory blocks both by
> the objects counters and by *valgrind*!
>
> In the result we have next data:
>
> Environment Initially, MB Iter. 1, MB Iter. 2, MB Iter. 3, MB Iter. 4,
> MB Iter. 5, MB Resume
> Debian 10 amd64, GLibC 2.28, GCC 8.3.0 182 191.5 199 206 212 212
> Satiated on the iteration*4*, base consumption 9.5 MB, extra consumption 20
> MB (*200* %), liboscada.so 3.5 MB, ui_WebVision.so 0.74 MB
> Debian 9 amd64, GLibC 2.24, GCC 6.3.0 160 170 178 179 183 185 Satiated
> on the iteration*5*, base consumption 10 MB, extra consumption 15 MB
> (*150* %), liboscada.so 3.5 MB, ui_WebVision.so 0.72 MB
> Debian 8 amd64, GLibC 2.19, GCC 4.9.2 125.5 133 139 139 139 139
> Satiated on the iteration*2*, base consumption 7.5 MB, extra consumption 6
> MB (*80* %), liboscada.so 3.8 MB, ui_WebVision.so 0.79 MB
> Debian 7 amd64, GLibC 2.13, GCC 4.7.2 101 108 111 112 112 112 Satiated
> on the iteration*2*, base consumption 7 MB, extra consumption 4 MB (*57* %),
> liboscada.so 3.4 MB, ui_WebVision.so 0.85 MB
> Debian 10 i386, GLibC 2.28, GCC 8.3.0 151 158 162.5 166 166 166
> Satiated on the iteration*3*, base consumption 7 MB, extra consumption 8 MB
> (*114* %), liboscada.so 3.7 MB, ui_WebVision.so 0.9 MB
> Debian 9 i386, GLibC 2.24, GCC 6.3.0 125 131 132 136 136 139 Satiated
> on the iteration*5*, base consumption 6 MB, extra consumption 8 MB
> (*133* %), liboscada.so 3.7 MB, ui_WebVision.so 0.9 MB
> Debian 8 i386, GLibC 2.19, GCC 4.9.2 92.5 99 101.5 103 103.5 103.5
> Satiated on the iteration*2*, base consumption 6.5 MB, extra consumption
> 4.5 (*69* %), liboscada.so 3.6 MB, ui_WebVision.so 0.94 MB
> Debian 7 i386, GLibC 2.13, GCC 4.7.2 70 76 76 76 77 77 Satiated on
> the iteration*2*, base consumption 6 MB, extra consumption 1 MB (*16* %),
> liboscada.so 3.6 MB, ui_WebVision.so 0.9 MB
> ALTLinux 6 i386, GLibC 2.11.3, GCC 4.5.4 69 74 75 75 75 75 Satiated on
> the iteration*2*, base consumption 5 MB, extra consumption 1 MB (*20* %),
> liboscada.so 2.3 MB, ui_WebVision.so 0.9 MB
>
> From the data we have the memory effectivity on AMD64 and I386 platform:
>
> and the absolute initial size for the both platform:
This indeed really shows an increase in memory consumption with the GNU
libc and the GCC versions. Have you tried to see if it comes mostly from
the GLIBC or from GCC? For example you can try to build your application
with GCC 7 on Debian 10. You can try to build your application on Debian
9 and run it on Debian 10 provided you do not have incompatible
libraries. Also do you use exactly the same versions of other libraries
in your tests?
> I know about "the Memory Allocation Tunables" (https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunables.html)
> and have tried them but:
> - I have not got any effect from environments like to
> "GLIBC_TUNABLES=glibc.malloc.trim_threshold=128" on Debian 10
GLIBC_TUNABLES should work on Debian 10. Now depending on the workload
you might see more or less effects.
> - If the tunables real work, why their do not apply globally (on the system
> level) to return the memory effectivity to the level of the Debian 7 (GLibC
> 2.13)?
Because every workload behave differently, and also not everybody cares
about the same. You seem to care about memory usage, some other care
about performance. The idea is to get a balanced memory allocator which
can be tuned.
> - If the new memory allocator (into GLibC 2.28) is so good, how can I return
> its memory effectivity to the level of the Debian 7 (GLibC 2.13)?
I have no idea about that, maybe playing with the other tunables. It's
also not impossible some of the increase is due to the security hardening
that has been enabled in debian over time.
Regards,
Aurelien
--
Aurelien Jarno GPG: 4096R/1DDD8C9B
aurelien@aurel32.net http://www.aurel32.net
Reply to: