Package: general
Version: AMD64
This is basically a debian AMD64-version of the bug report for ubuntu AMD64 bug 24691 [1].
The problem is (seems to be) that a lot of libraries are built with alignment above 2**3. Most of these cases are actually 2**20 in ubuntu AMD64 and 2**21 in debian AMD64. In other words, 1 and 2 MB correspondingly!
I often see loose and vague arguments such as "if 99MB of that is shared, the calculator is really only 'using' 1MB of ram -- and that's fine" [2].
It's not fine. And it's certainly incorrect. Just because a library is shared doesn't mean it's fine that it consumes megabytes(!) of memory in vain. Especially libraries that are "shared" "between" 1 process.
Now, I can't figure out why such huge amounts of memory is hogged on my computer. But I need to restart X about once a week. If I don't, my 4 GB of RAM is quickly filled and my 4 GB swap starts to work (hurray, 15 second delay when changing virtual desktop). It's been like this since I bought this machine (soon 2 years ago), and I frequently update my X driver (-radeon, -radeonhd, fglrx, etc), so I doubt they are to blame for stealing my memory, even though it could've been a good guess.
memstat reports lots and lots of libraries which consumes slightly more than 2 mb each:
$ memstat | grep '\.so' | grep -v PID
gives me 551 lines, and by just grasping the result, easily 90% of them are slightly more than 2 mb (2**21 + small stuff).
If these libraries could be built with 2**3 (8 byte) alignment, instead of 2**21 (2 megabytes), I assume, just like the discussion in [1], that at least some memory wouldn't be wasted in vain.
So how many libraries (on my system) are built with 2**21 alignment?
/lib: $ for file in `\ls *.so.*` ; do if objdump -x $file | grep -q -e '2\*\*21' ; then echo $file ; fi ; done | wc
99
/usr/lib: $ for file in `\ls *.so.*` ; do if objdump -x $file | grep -q -e '2\*\*21' ; then echo $file ; fi ; done | wc
2777
/usr/lib/*: $ for file in `\ls */*.so.*` ; do if objdump -x $file | grep -q -e '2\*\*21' ; then echo $file ; fi ; done | wc
396
99 + 2777 + 396 = 3272.
Quite a lot of libraries. Loading them all would require roughly 7 GB. Remember: loading. Not using. And these are just on my system, it's not even close to "all" libraries in debian.
Just looking at how much memory pidgin-specific plugins consumes is frightening:
memstat | grep -E '(purple|pidgin)+.*\.so.*'
Returns 88 libraries, ALL consuming slightly more than 2 mb. How many of them are "shared" with any other program than pidgin? I'd say none. But that's just a guess.
So this means at least 200 mb memory usage for pidgin alone? Could this really be the case?!
$ ps aux | grep pidgin
gustaf 30432 0.2 1.7 631196 68532 ? S Aug26 2:36 pidgin
Oh yeah, 631192 kB virtual and 68532 kB resident.
I say this again, some people argues; "but most of that 616 MB is shared so it doesn't matter". It matters, because Linux prefers to swap it, to give place for IO buffers, and when things are being swapped, holy moses, Ctrl+Alt+Backspace is thy saviour.
Pidgin consuming 616 MB virtual memory is just.. Well.. Messed up. To put it lightly.
It could be reasonable to see the rest of the memory hoggers on my system, to make it clear that this really is a big problem.
Please note that pidgin is only on 8th place! This is 'top' sorted by 'M' (memory usage):
31095 gustaf 20 0 1314m 700m 35m R 30 17.8 252:06.36 firefox-bin
32197 gustaf 20 0 1006m 439m 39m S 12 11.2 167:03.50 epiphany-browse
30134 root 20 0 625m 256m 14m S 8 6.5 99:41.51 Xorg
2680 gustaf 20 0 479m 103m 25m S 0 2.6 4:07.31 banshee-1
30305 gustaf 20 0 438m 82m 17m S 0 2.1 0:31.87 /usr/lib/ontv/o
1621 gustaf 20 0 304m 76m 14m S 0 1.9 0:42.91 gnome-terminal
6070 clamav 20 0 92524 76m 380 S 0 1.9 0:00.00 clamd
30432 gustaf 20 0 616m 66m 28m S 1 1.7 2:37.08 pidgin
$ free -m
total used free shared buffers cached
Mem: 3934 3891 43 0 331 1186
-/+ buffers/cache: 2373 1561
Swap: 3859 5 3853
Real memory usage of my very recently booted machine (I haven't started even a small subset of the apps I usually run): 2373 MB!
$ uptime
12:27:24 up 21:44, 13 users, load average: 1.09, 1.05, 0.99
I'm not sure my mobo supports much more than what I have, but I'm seriously thinking of spending (too much) money on 8 or 16 GB RAM, just to have a system that is usable.
Oh yeah, I'm using 'sid', maybe that's important to know, by I certainly doubt it.
I haven't seen many discussions about this in debian, and just 1 about ubuntu [1]. It could be that I have gotten things completely the wrong way (then please tell me so), but it could also be that this has been overseen for too long, and now needs serious attention.
Memory is cheap nowadays, but it isn't free. And if we're expected to buy dozens of GB to get a usable system, then, well, debian is getting rather expensive I'm afraid.
Regards,
Gustaf
[1] https://bugs.launchpad.net/ubuntu/+bug/24692
[2] https://lists.ubuntu.com/archives/ubuntu-devel/2005-October/012653.html