Re: Need help analyzing (kernel?) memory usage and reclaiming RAM (Debian Stretch)
Hi.
On Tue, Apr 16, 2019 at 02:30:57PM +0200, Martin Schwarz wrote:
> > slabtop from "procps" package, definitely.
> > Should've thought of it earlier.
>
> I did take a look at slaptop (and at /proc/slabinfo) before. But since the
> values for "SReclaimable" and "SUnreclaim" from /proc/meminfo are not high
> enough to explain the memory consumption, I didn't investigate any further in
> that direction.
>
> Looks unsuspicious to me:
Agreed. Nothing here that explains the behaviour observed.
> ['perf top' and 'perf record']
> > It's very indirect method. Basically it shows internal kernel functions
> > used, which may or may not be the source of the leak.
>
> I'm afraid 'perf' is beyond my knowledge. In its default invocation, 'perf_4.9
> top' seems to be more focussed on CPU usage?
Nope, it's deeper than this. What "perf top" should show by default is
the kernel and userspace library functions called by every kernel thread
and every userspace process. For instance,
81.98% [kernel] [k] load_balance
18.02% [kernel] [k] __tick_nohz_idle_enter
0.00% [kernel] [k] module_get_kallsym
This shows mostly idle OS, and the kernel calls load_balance() 81% of
the time, and __tick_nohz_idle_enter() 18% of the time. It eats CPU, of
course, but CPU consumption is not a concern here. The answer to the
question 'what the kernel does' is.
> And 'perf_4.9 record' is used to profile a certain command?
'perf record -a' in this case. All userspace and the kernel.
Terminate it with Ctrl+C after a minute or two.
Sorry if I didn't mention that.
> Tried with "sleep 30" as command, but not sure how
> to interpret the recording.
'perf report' from the same directory where it wrote "pref.data"
earlier.
> So I'm really unsure how to use them to further drill into kernel or module
> memory usage. Any hints?
Send it here, unless it's classified or somehow private. You'll need
those anyway, because...
>
> > There's this SystemTap thing that can presumably do it better, but last
> > time I've checked it required a debug version of the kernel (and that's
> > unsuitable for just about anything short of kernel development).
>
> By then I should probably go to debian-kernel? ;-)
I'd file a bug against linux-image package. You're using only in-tree
kernel modules shipped by Debian. The kernel is obviously misbehaving.
And if they won't answer you in a week (it's a small volunteer team
after all) - then you'll go at debian-kernel with a bug number at hands.
Reco
Reply to: