[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#934080: [libc6] Significant degradation in the memory effectivity of the memory allocator



Hello, Aurelien Jarno

06.08.19 23:57, Aurelien Jarno wrote:

The live disks were started under VirtualBox 5.2, where the got data was
measured by *top*.
Can you details more precisely on you measure the memory used? Do you
just get the line corresponding to the process you want to monitor?
Sure
Which column do you take?

"RES", sure


liboscada.so 2.3 MB, ui_WebVision.so 0.9 MB

>From the data we have the memory effectivity on AMD64 and I386 platform:

and the absolute initial size for the both platform:
This indeed really shows an increase in memory consumption with the GNU
libc and the GCC versions. Have you tried to see if it comes mostly from
the GLIBC or from GCC? 

No, but I thought initially about GCC, and it version included to this table.

After deep familiarizing the problem I saw the GCC of the different version build mostly equal in the size binaries and the memory allocator is a part of GLibC as the functions malloc(), realloc(), free().

For example you can try to build your application
with GCC 7 on Debian 10.
This I am going to try.
 You can try to build your application on Debian
9 and run it on Debian 10 provided you do not have incompatible
libraries. Also do you use exactly the same versions of other libraries
in your tests?

I have used libraries of the corresponded distributive which does not influence to the relative and final extra memory consumption, at least.

And that is not possible:

I know about "the Memory Allocation Tunables" (https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunables.html)
and have tried them but:
- I have not got any effect from environments like to
"GLIBC_TUNABLES=glibc.malloc.trim_threshold=128" on Debian 10
GLIBC_TUNABLES should work on Debian 10. Now depending on the workload
you might see more or less effects.
Sure, it is near the measuring method error and far from the Debian 7 values.
- If the tunables real work, why their do not apply globally (on the system
level) to return the memory effectivity to the level of the Debian 7 (GLibC
2.13)?
Because every workload behave differently, and also not everybody cares
about the same. You seem to care about memory usage, some other care
about performance. The idea is to get a balanced memory allocator which
can be tuned.

This is only a representative example, and in the real life this problem is far worse.

I have a real-task environment on Debian 9 which consume initially about 1.6 GB and after a pair days of such work it consume about 6GB!

In other hand, I also have an old environment on Debian 7 which consumes very small extra memory, really frees the consumed memory and works the same fast.

- If the new memory allocator (into GLibC 2.28) is so good, how can I return
its memory effectivity to the level of the Debian 7 (GLibC 2.13)?
I have no idea about that, maybe playing with the other tunables. 

It is not possible, or if you show me some real working examples, I will try.

It's
also not impossible some of the increase is due to the security hardening
that has been enabled in debian over time.

So, we have got such regression, and I have to think about back-using Debian 7 on such sort dynamic environments and forget all new ones. :(

Regards, Roman


Reply to: