"page allocation failure" errors
Hello,
My debian stable system has been spewing "page allocation failure"
errors since the past few months. I have attached a sample error log.
This does not occur too frequently. Sometimes, my system functions
normally for several weeks before these errors crop up. When they do, my
system is brought to its knees swapping furiously. Many times, a hard
reset was the only option.
Google led me to bugs 533180 and 572932 but I am not sure if my problem
is due to those. Both the stock lenny kernel as well as the backports
kernel give these messages. How do I go about identifying the source of
these errors? How do I determine if it is a hardware issue or a software
one? BTW memtest did not report anything.
My system is an AMD 64bit dual core, dual processor with 8GB ram and an
Nvidia graphics card with the proprietary driver. I had been running
lenny stable since its release without any issues.
Thank you,
Raj Kiran
Sep 29 09:11:16 rajkiran kernel: [1377549.301678] swapper: page allocation failure. order:3, mode:0x20
Sep 29 09:11:16 rajkiran kernel: [1377549.301689] Pid: 0, comm: swapper Tainted: P 2.6.30-bpo.1-amd64 #1
Sep 29 09:11:16 rajkiran kernel: [1377549.301695] Call Trace:
Sep 29 09:11:16 rajkiran kernel: [1377549.301700] <IRQ> [<ffffffff80297e92>] ? __alloc_pages_internal+0x404/0x426
Sep 29 09:11:16 rajkiran kernel: [1377549.301732] [<ffffffff802bb0ec>] ? kmem_getpages+0x68/0x12b
Sep 29 09:11:16 rajkiran kernel: [1377549.301740] [<ffffffff802bb302>] ? fallback_alloc+0x153/0x1cd
Sep 29 09:11:16 rajkiran kernel: [1377549.301747] [<ffffffff802bb48c>] ? ____cache_alloc_node+0x110/0x12f
Sep 29 09:11:16 rajkiran kernel: [1377549.301755] [<ffffffff802bb556>] ? kmem_cache_alloc_node+0xab/0x11e
Sep 29 09:11:16 rajkiran kernel: [1377549.301762] [<ffffffff802bb625>] ? __kmalloc_node+0x5c/0xa7
Sep 29 09:11:16 rajkiran kernel: [1377549.301773] [<ffffffff8040b430>] ? __alloc_skb+0x66/0x138
Sep 29 09:11:16 rajkiran kernel: [1377549.301827] [<ffffffffa0062838>] ? pci_map_single+0x20/0x8d [forcedeth]
Sep 29 09:11:16 rajkiran kernel: [1377549.301837] [<ffffffff8040bc8f>] ? skb_copy+0x30/0x96
Sep 29 09:11:16 rajkiran kernel: [1377549.301851] [<ffffffffa10a5411>] ? vboxNetFltLinuxPacketHandler+0x40/0x95 [vboxnetflt]
Sep 29 09:11:16 rajkiran kernel: [1377549.301863] [<ffffffff8041228d>] ? dev_hard_start_xmit+0x140/0x272
Sep 29 09:11:16 rajkiran kernel: [1377549.301873] [<ffffffff80423b09>] ? __qdisc_run+0xe2/0x1ed
Sep 29 09:11:16 rajkiran kernel: [1377549.301881] [<ffffffff804126f0>] ? dev_queue_xmit+0x230/0x32a
Sep 29 09:11:16 rajkiran kernel: [1377549.301893] [<ffffffff80437457>] ? ip_queue_xmit+0x2d4/0x330
Sep 29 09:11:16 rajkiran kernel: [1377549.301904] [<ffffffff802ba8e3>] ? __drain_alien_cache+0x8f/0xa1
Sep 29 09:11:16 rajkiran kernel: [1377549.301915] [<ffffffff80447a47>] ? tcp_transmit_skb+0x63e/0x67d
Sep 29 09:11:16 rajkiran kernel: [1377549.301923] [<ffffffff80449ecc>] ? tcp_write_xmit+0x874/0x96c
Sep 29 09:11:16 rajkiran kernel: [1377549.301932] [<ffffffff802ba8e3>] ? __drain_alien_cache+0x8f/0xa1
Sep 29 09:11:16 rajkiran kernel: [1377549.301940] [<ffffffff8044a011>] ? __tcp_push_pending_frames+0x22/0x53
Sep 29 09:11:16 rajkiran kernel: [1377549.301948] [<ffffffff8044538f>] ? tcp_data_snd_check+0x21/0xf2
Sep 29 09:11:16 rajkiran kernel: [1377549.301956] [<ffffffff8044606a>] ? tcp_rcv_established+0x6a8/0x711
Sep 29 09:11:16 rajkiran kernel: [1377549.301966] [<ffffffff8044ce9d>] ? tcp_v4_do_rcv+0x1ab/0x34f
Sep 29 09:11:16 rajkiran kernel: [1377549.301975] [<ffffffff80430d6e>] ? ip_route_input+0x58/0xd17
Sep 29 09:11:16 rajkiran kernel: [1377549.301982] [<ffffffff80439c41>] ? __inet_lookup_established+0x43/0x20c
Sep 29 09:11:16 rajkiran kernel: [1377549.301991] [<ffffffff8044d4cb>] ? tcp_v4_rcv+0x48a/0x6db
Sep 29 09:11:16 rajkiran kernel: [1377549.302000] [<ffffffff804330d8>] ? ip_local_deliver_finish+0x146/0x1e9
Sep 29 09:11:16 rajkiran kernel: [1377549.302008] [<ffffffff80432c75>] ? ip_rcv_finish+0x321/0x33b
Sep 29 09:11:16 rajkiran kernel: [1377549.302017] [<ffffffff804119ea>] ? process_backlog+0x81/0xb3
Sep 29 09:11:16 rajkiran kernel: [1377549.302025] [<ffffffff8040fc83>] ? net_rx_action+0xa8/0x191
Sep 29 09:11:16 rajkiran kernel: [1377549.302043] [<ffffffffa0065012>] ? nv_nic_irq_optimized+0xe5/0x25b [forcedeth]
Sep 29 09:11:16 rajkiran kernel: [1377549.302054] [<ffffffff80246f91>] ? __do_softirq+0xac/0x173
Sep 29 09:11:16 rajkiran kernel: [1377549.302064] [<ffffffff80210bcc>] ? call_softirq+0x1c/0x30
Sep 29 09:11:16 rajkiran kernel: [1377549.302072] [<ffffffff802125fa>] ? do_softirq+0x3a/0x7e
Sep 29 09:11:16 rajkiran kernel: [1377549.302079] [<ffffffff80246d0e>] ? irq_exit+0x3f/0x80
Sep 29 09:11:16 rajkiran kernel: [1377549.302086] [<ffffffff80211ece>] ? do_IRQ+0xa9/0xbf
Sep 29 09:11:16 rajkiran kernel: [1377549.302093] [<ffffffff80210453>] ? ret_from_intr+0x0/0x11
Sep 29 09:11:16 rajkiran kernel: [1377549.302098] <EOI> [<ffffffff80256fa9>] ? hrtimer_cancel+0xc/0x16
Sep 29 09:11:16 rajkiran kernel: [1377549.302115] [<ffffffff80227520>] ? native_safe_halt+0x2/0x3
Sep 29 09:11:16 rajkiran kernel: [1377549.302124] [<ffffffff80216995>] ? default_idle+0x40/0x68
Sep 29 09:11:16 rajkiran kernel: [1377549.302133] [<ffffffff80257d0d>] ? notifier_call_chain+0x29/0x4c
Sep 29 09:11:16 rajkiran kernel: [1377549.302141] [<ffffffff80216d6a>] ? c1e_idle+0x107/0x10d
Sep 29 09:11:16 rajkiran kernel: [1377549.302151] [<ffffffff8020edda>] ? cpu_idle+0x50/0x91
Sep 29 09:11:16 rajkiran kernel: [1377549.302156] Mem-Info:
Sep 29 09:11:16 rajkiran kernel: [1377549.302161] Node 0 DMA per-cpu:
Sep 29 09:11:16 rajkiran kernel: [1377549.302167] CPU 0: hi: 0, btch: 1 usd: 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302173] CPU 1: hi: 0, btch: 1 usd: 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302178] CPU 2: hi: 0, btch: 1 usd: 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302183] CPU 3: hi: 0, btch: 1 usd: 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302187] Node 0 DMA32 per-cpu:
Sep 29 09:11:16 rajkiran kernel: [1377549.302193] CPU 0: hi: 186, btch: 31 usd: 178
Sep 29 09:11:16 rajkiran kernel: [1377549.302198] CPU 1: hi: 186, btch: 31 usd: 14
Sep 29 09:11:16 rajkiran kernel: [1377549.302203] CPU 2: hi: 186, btch: 31 usd: 183
Sep 29 09:11:16 rajkiran kernel: [1377549.302208] CPU 3: hi: 186, btch: 31 usd: 25
Sep 29 09:11:16 rajkiran kernel: [1377549.302212] Node 0 Normal per-cpu:
Sep 29 09:11:16 rajkiran kernel: [1377549.302218] CPU 0: hi: 186, btch: 31 usd: 155
Sep 29 09:11:16 rajkiran kernel: [1377549.302222] CPU 1: hi: 186, btch: 31 usd: 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302228] CPU 2: hi: 186, btch: 31 usd: 87
Sep 29 09:11:16 rajkiran kernel: [1377549.302233] CPU 3: hi: 186, btch: 31 usd: 170
Sep 29 09:11:16 rajkiran kernel: [1377549.302237] Node 1 Normal per-cpu:
Sep 29 09:11:16 rajkiran kernel: [1377549.302243] CPU 0: hi: 186, btch: 31 usd: 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302248] CPU 1: hi: 186, btch: 31 usd: 166
Sep 29 09:11:16 rajkiran kernel: [1377549.302253] CPU 2: hi: 186, btch: 31 usd: 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302258] CPU 3: hi: 186, btch: 31 usd: 98
Sep 29 09:11:16 rajkiran kernel: [1377549.302268] Active_anon:148159 active_file:68722 inactive_anon:124575
Sep 29 09:11:16 rajkiran kernel: [1377549.302271] inactive_file:96462 unevictable:169909 dirty:23 writeback:0 unstable:0
Sep 29 09:11:16 rajkiran kernel: [1377549.302274] free:175200 slab:1217090 mapped:31928 pagetables:7135 bounce:0
Sep 29 09:11:16 rajkiran kernel: [1377549.302280] Node 0 DMA free:9876kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:8788kB pages_scanned:0 all_unreclaimable? yes
Sep 29 09:11:16 rajkiran kernel: [1377549.302294] lowmem_reserve[]: 0 3511 4016 4016
Sep 29 09:11:16 rajkiran kernel: [1377549.302302] Node 0 DMA32 free:663648kB min:5000kB low:6248kB high:7500kB active_anon:9228kB inactive_anon:93564kB active_file:261384kB inactive_file:331528kB unevictable:216476kB present:3596284kB pages_scanned:12 all_unreclaimable? no
Sep 29 09:11:16 rajkiran kernel: [1377549.302317] lowmem_reserve[]: 0 0 505 505
Sep 29 09:11:16 rajkiran kernel: [1377549.302325] Node 0 Normal free:3500kB min:716kB low:892kB high:1072kB active_anon:45248kB inactive_anon:76264kB active_file:852kB inactive_file:2988kB unevictable:62272kB present:517120kB pages_scanned:0 all_unreclaimable? no
Sep 29 09:11:16 rajkiran kernel: [1377549.302339] lowmem_reserve[]: 0 0 0 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302346] Node 1 Normal free:23776kB min:5756kB low:7192kB high:8632kB active_anon:538160kB inactive_anon:328472kB active_file:12652kB inactive_file:51332kB unevictable:400888kB present:4136960kB pages_scanned:65 all_unreclaimable? no
Sep 29 09:11:16 rajkiran kernel: [1377549.302361] lowmem_reserve[]: 0 0 0 0
Sep 29 09:11:16 rajkiran kernel: [1377549.302368] Node 0 DMA: 3*4kB 3*8kB 3*16kB 2*32kB 4*64kB 4*128kB 3*256kB 2*512kB 1*1024kB 1*2048kB 1*4096kB = 9876kB
Sep 29 09:11:16 rajkiran kernel: [1377549.302388] Node 0 DMA32: 98862*4kB 23885*8kB 4824*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 663744kB
Sep 29 09:11:16 rajkiran kernel: [1377549.302409] Node 0 Normal: 807*4kB 32*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3500kB
Sep 29 09:11:16 rajkiran kernel: [1377549.302429] Node 1 Normal: 5855*4kB 9*8kB 13*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 23700kB
Sep 29 09:11:16 rajkiran kernel: [1377549.302449] 233593 total pagecache pages
Sep 29 09:11:16 rajkiran kernel: [1377549.302453] 66142 pages in swap cache
Sep 29 09:11:16 rajkiran kernel: [1377549.302458] Swap cache stats: add 975246, delete 909104, find 11553445/11660238
Sep 29 09:11:16 rajkiran kernel: [1377549.302463] Free swap = 9751924kB
Sep 29 09:11:16 rajkiran kernel: [1377549.302467] Total swap = 10490436kB
Sep 29 09:11:16 rajkiran kernel: [1377549.305587] 2097152 pages RAM
Sep 29 09:11:16 rajkiran kernel: [1377549.305587] 68604 pages reserved
Sep 29 09:11:16 rajkiran kernel: [1377549.305587] 451201 pages shared
Sep 29 09:11:16 rajkiran kernel: [1377549.305587] 1252404 pages non-shared
Reply to: