Re: lenny kernel memory leak?
On Thu, Dec 17, 2009 at 01:41:16AM +0000, Ben Hutchings wrote:
> Did you see any repeated error messages in the kernel log?
Yes, rt2500 related debug messages which I'm unable to turn off:
[235548.961171] wlan0: Selected IBSS BSSID 02:0c:f1:46:6a:78 based on configured SSID
[235548.961265] wlan0: Configured IBSS beacon template
[235578.961375] wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge)
[235579.821460] wlan0: Selected IBSS BSSID 02:0c:f1:46:bc:cb based on configured SSID
[235579.821598] wlan0: Configured IBSS beacon template
[235609.821427] wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge)
[235610.681582] wlan0: Selected IBSS BSSID 02:0c:f1:46:6a:78 based on configured SSID
[235610.681688] wlan0: Configured IBSS beacon template
wlan0 is in ad-hoc mode. In the OOM cases I briefly saw the killer in action
with logs like:
Dec 5 21:40:23 nalle kernel: [423710.429160] wlan0: No active IBSS STAs - tryin
g to scan for other IBSS networks with same SSID (merge)
Dec 5 21:40:24 nalle kernel: [423711.289415] wlan0: Selected IBSS BSSID 02:0c:f
1:46:cb:2f based on configured SSID
Dec 5 21:40:24 nalle kernel: [423711.289617] wlan0: Configured IBSS beacon temp
late
Dec 5 21:40:41 nalle kernel: [423726.661954] ssh invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
Dec 5 21:40:41 nalle kernel: [423726.662085] Pid: 2512, comm: ssh Not tainted 2.6.26-2-486 #1
Dec 5 21:40:41 nalle kernel: [423726.662244] [<c014cea8>] oom_kill_process+0x4a/0x168
Dec 5 21:40:41 nalle kernel: [423726.662485] [<c014d246>] out_of_memory+0x133/0x15d
Dec 5 21:40:41 nalle kernel: [423726.662719] [<c014ed7b>] __alloc_pages_internal+0x296/0x32c
Dec 5 21:40:41 nalle kernel: [423726.662963] [<c014ee1d>] __alloc_pages+0x7/0x9
Dec 5 21:40:41 nalle kernel: [423726.663052] [<c015062a>] __do_page_cache_readahead+0x6e/0x14a
Dec 5 21:40:41 nalle kernel: [423726.663249] [<c0150a29>] do_page_cache_readahead+0x3d/0x48
Dec 5 21:40:41 nalle kernel: [423726.663394] [<c014c70f>] filemap_fault+0x13c/0x2fd
Dec 5 21:40:41 nalle kernel: [423726.663632] [<c015489f>] __do_fault+0x42/0x2f2
Dec 5 21:40:41 nalle kernel: [423726.663716] [<c0154f2a>] do_wp_page+0x3db/0x434
Dec 5 21:40:41 nalle kernel: [423726.663986] [<c0155dcf>] handle_mm_fault+0x28e/0x5d2
Dec 5 21:40:41 nalle kernel: [423726.664374] [<c0113b6d>] do_page_fault+0x29c/0x5ef
Dec 5 21:40:41 nalle kernel: [423726.664535] [<c01663f3>] sys_read+0x3c/0x63
Dec 5 21:40:41 nalle kernel: [423726.664660] [<c01138d1>] do_page_fault+0x0/0x5ef
Dec 5 21:40:41 nalle kernel: [423726.664746] [<c02a74ba>] error_code+0x6a/0x70
Dec 5 21:40:41 nalle kernel: [423726.665060] =======================
Dec 5 21:40:41 nalle kernel: [423726.665107] Mem-info:
Dec 5 21:40:41 nalle kernel: [423726.665148] DMA per-cpu:
Dec 5 21:40:41 nalle kernel: [423726.665196] CPU 0: hi: 0, btch: 1 usd: 0
Dec 5 21:40:41 nalle kernel: [423726.665250] Normal per-cpu:
Dec 5 21:40:41 nalle kernel: [423726.665296] CPU 0: hi: 0, btch: 1 usd: 0
Dec 5 21:40:41 nalle kernel: [423726.665370] Active:154 inactive:472 dirty:0 writeback:0 unstable:0
Dec 5 21:40:41 nalle kernel: [423726.665387] free:210 slab:5440 mapped:72 pagetables:160 bounce:0
Dec 5 21:40:41 nalle kernel: [423726.665509] DMA free:420kB min:360kB low:448kB high:540kB active:616kB inactive:1860kB present:16256kB pages_scanned:186 all_unreclaimable? yes
Dec 5 21:40:41 nalle kernel: [423726.665660] lowmem_reserve[]: 0 15 15
Dec 5 21:40:41 nalle kernel: [423726.665755] Normal free:420kB min:360kB low:448kB high:540kB active:0kB inactive:28kB present:16256kB pages_scanned:7 all_unreclaimable? no
Dec 5 21:40:41 nalle kernel: [423726.665868] lowmem_reserve[]: 0 0 0
Dec 5 21:40:41 nalle kernel: [423726.665934] DMA: 1*4kB 0*8kB 0*16kB 1*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 420kB
Dec 5 21:40:41 nalle kernel: [423726.666115] Normal: 13*4kB 2*8kB 2*16kB 0*32kB 1*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 420kB
Dec 5 21:40:41 nalle kernel: [423726.666298] 626 total pagecache pages
Dec 5 21:40:41 nalle kernel: [423726.666353] Swap cache: add 348409, delete 348295, find 105774/174184
Dec 5 21:40:41 nalle kernel: [423726.666420] Free swap = 182440kB
Dec 5 21:40:41 nalle kernel: [423726.666468] Total swap = 204792kB
Dec 5 21:40:41 nalle kernel: [423726.668597] 8192 pages of RAM
Dec 5 21:40:41 nalle kernel: [423726.668644] 0 pages of HIGHMEM
Dec 5 21:40:41 nalle kernel: [423726.668686] 885 reserved pages
Dec 5 21:40:41 nalle kernel: [423726.668727] 142 pages shared
Dec 5 21:40:41 nalle kernel: [423726.668768] 114 pages swap cached
Dec 5 21:40:41 nalle kernel: [423726.668811] 0 pages dirty
Dec 5 21:40:41 nalle kernel: [423726.668850] 0 pages writeback
Dec 5 21:40:41 nalle kernel: [423726.668891] 72 pages mapped
Dec 5 21:40:41 nalle kernel: [423726.668931] 5440 pages slab
Dec 5 21:40:41 nalle kernel: [423726.668971] 160 pages pagetables
Dec 5 21:40:41 nalle kernel: [423726.669024] Out of memory: kill process 9948 (sshd) score 2874 or a child
Dec 5 21:40:41 nalle kernel: [423726.669125] Killed process 9949 (bash)
Dec 5 21:40:46 nalle kernel: [423732.788333] phy0: Adding new IBSS station 00:11:09:9a:bd:3a (dev=wlan0)
Dec 5 21:40:54 nalle kernel: [423741.290156] wlan0: expiring inactive STA 00:11:09:9a:bd:3a
Dec 5 21:40:54 nalle kernel: [423741.290785] wlan0: No active IBSS STAs - trying
Since then I have switched system shell from bash to dash, shut down a few
non-essential services and mounted all fs with noatime. These have helped a
bit but I still see slab growing about 1 mb in day.
$ uptime
08:01:02 up 2 days, 17:35, 3 users, load average: 0.11, 0.11, 0.06
$ grep Slab /proc/meminfo
Slab: 6448 kB
> In particular, did you see "Failed to send RTS/CTS frame"? A later
> verison of the kernel fixes a memory leak associated with that specific
> error case, and I may be able to apply that fix.
This I have not seen in the logs. Perhaps it's because I'm using ad-hoc mode.
I didn't succeed in using rt2500pci in AP mode with lenny kernel and hostapd.
-Mikko
Reply to: