Re: Out of memory killer misconfigured?
On 20/04/2022 17:11, Jonathan Dowland wrote:
On Wed, Apr 20, 2022 at 04:23:38PM +0100, piorunz wrote:
Sorry but this happened to me a few times, each tome with the same Wine
program. it just loves to eat all available memory when its doing heavy
computations. When I am not careful and I start too many threads, is
eats all memory. My system gets killed every single time. I am not
extrapolating anything. Misbehaving app should get killed, but instead,
Linux kills itself.
Ok, not one experience, but one scenario. And there are plausible
reasons why this might be happening as outlined in my other mail to the
thread (OOM adjustments to make the killer skip over the process).
Occam's razor suggests something about your particular setup versus
the OOM killer simply being as bad as you think it is. FWIW, I invoke
the OOM killer a lot recently (due to some scientific experiments) and
the sensible processes were killed in my case, every time, leaving my
desktop functional.
More data about your setup is needed to fathom out what's going on.
I didn't configured OOM. I use default Debian Testing with KDE. I
changed nothing apart from user desktop things. MY situation is not "a
scenario". It's what everyone can reproduce, just install Wine and
MetaTester5 program, I can guide you. I cannot guarantee this will
happen with non-Wine programs.
So I just reproduced this again, system crashed totally because I forgot
to apply choom --adjust 1000 to all metatester processes. Reboot. After
reboot, I started MT again and applied the following:
renice 19 `pidof metatester64.ex`
choom -p `pidof metatester64.ex | awk {'print $1'}` --adjust 1000
choom -p `pidof metatester64.ex | awk {'print $2'}` --adjust 1000
choom -p `pidof metatester64.ex | awk {'print $3'}` --adjust 1000
choom -p `pidof metatester64.ex | awk {'print $4'}` --adjust 1000
choom -p `pidof metatester64.ex | awk {'print $5'}` --adjust 1000
choom -p `pidof metatester64.ex | awk {'print $6'}` --adjust 1000
choom -p `pidof metatester64.ex | awk {'print $7'}` --adjust 1000
I spawned more than 8 processes to saturate RAM quickly. Meaning, some
processes were not adjusted to 1000 score. After 20 seconds, OOM event
happened.
And what Linux does? It has at least 8 processes with 1000 score, but
let's see what is does:
$ sudo dmesg | grep "Out of memory"
[ 704.018238] Out of memory: Killed process 39674 (metatester64.ex)
total-vm:2939256kB, anon-rss:250524kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:1344kB oom_score_adj:1000
[ 708.528663] Out of memory: Killed process 25882 (QtWebEngineProc)
total-vm:5451920kB, anon-rss:20588kB, file-rss:0kB, shmem-rss:116kB,
UID:1000 pgtables:772kB oom_score_adj:300
[ 709.811945] Out of memory: Killed process 23411 (krunner-keepass)
total-vm:52680kB, anon-rss:12860kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:140kB oom_score_adj:200
[ 710.788999] Out of memory: Killed process 23413 (pipewire-media-)
total-vm:170052kB, anon-rss:9492kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:248kB oom_score_adj:200
[ 712.748691] Out of memory: Killed process 23618 (kactivitymanage)
total-vm:550696kB, anon-rss:8848kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:244kB oom_score_adj:200
[ 714.856773] Out of memory: Killed process 23414 (pulseaudio)
total-vm:1384772kB, anon-rss:8508kB, file-rss:0kB, shmem-rss:260kB,
UID:1000 pgtables:320kB oom_score_adj:200
[ 716.051274] Out of memory: Killed process 23560 (kglobalaccel5)
total-vm:282368kB, anon-rss:6728kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:256kB oom_score_adj:200
[ 716.082495] Out of memory: Killed process 23412 (pipewire)
total-vm:53124kB, anon-rss:4484kB, file-rss:0kB, shmem-rss:52kB,
UID:1000 pgtables:104kB oom_score_adj:200
[ 717.965930] Out of memory: Killed process 23867 (ksystemstats)
total-vm:166540kB, anon-rss:3844kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:176kB oom_score_adj:200
[ 718.775772] Out of memory: Killed process 23703 (kscreen_backend)
total-vm:227996kB, anon-rss:3060kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:164kB oom_score_adj:200
[ 718.792339] Out of memory: Killed process 23417 (dbus-daemon)
total-vm:10552kB, anon-rss:1332kB, file-rss:0kB, shmem-rss:0kB, UID:1000
pgtables:60kB oom_score_adj:200
[ 719.422552] Out of memory: Killed process 23714 (obexd)
total-vm:45836kB, anon-rss:732kB, file-rss:0kB, shmem-rss:0kB, UID:1000
pgtables:76kB oom_score_adj:200
[ 719.423354] Out of memory: Killed process 23555 (dconf-service)
total-vm:157600kB, anon-rss:544kB, file-rss:0kB, shmem-rss:0kB, UID:1000
pgtables:64kB oom_score_adj:200
[ 719.424525] Out of memory: Killed process 23397 ((sd-pam))
total-vm:169688kB, anon-rss:3724kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:92kB oom_score_adj:100
[ 719.425356] Out of memory: Killed process 23396 (systemd)
total-vm:18600kB, anon-rss:1912kB, file-rss:0kB, shmem-rss:0kB, UID:1000
pgtables:76kB oom_score_adj:100
[ 719.426070] Out of memory: Killed process 55808 (metatester64.ex)
total-vm:5510844kB, anon-rss:3909364kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:8516kB oom_score_adj:0
I lost Metatester (as expected), but I also lost sound and shortcuts and
KDE widgets. I need to reboot.
Look at memory sizes for last process and other processes killed. Last one:
Out of memory: Killed process 55808 (metatester64.ex)
total-vm:5510844kB, anon-rss:3909364kB, file-rss:0kB, shmem-rss:0kB,
UID:1000 pgtables:8516kB oom_score_adj:0
Why this is not killed right after 39674 (metatester64.ex)?
Why other 7 metatester64.ex processes for which I adjusted priority to
1000 were not killed before KDE processes were killed?
--
With kindest regards, Piotr.
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀⠀⠀
Reply to: