[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#686847: KVM guest slows down and finally hangs on I/O



Package: linux-2.6
Version: 2.6.32-45
Severity: normal

Linux vmXXXX.cs16.seeweb.it 2.6.32-5-amd64 #1 SMP Sun May 6 04:00:17 UTC 2012 x86_64 GNU/Linux

This is a fully up to date squeeze system running as a guest on a RHEL 
6.2 host. 1 CPU, 1 GB of RAM, virtio net and disk (using XFS).
We have hundreds of similar guests but only this one shows this problem.

The system suddenly slows down and then all processes hang. The getty on 
the virtual console is responsive to input, but it hangs when it tries 
to fork+exec login.

The workload is unmemorable, apparently mostly light apache/PHP.

The kernel logs this same stack trace for all (?) processes:

INFO: task apache2:12623 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
apache2       D 0000000000000000     0 12623  12619 0x00000000
 ffffffff814611f0 0000000000000086 0000000000000292 ffffffff8118ac27
 ffff88002faf8000 000000108102cdc4 000000000000f9e0 ffff880003113fd8
 0000000000015780 0000000000015780 ffff88002faf8000 ffff88002faf82f8
Call Trace:
 [<ffffffff8118ac27>] ? cfq_set_request+0x2af/0x319
 [<ffffffff8102cdc4>] ? pvclock_clocksource_read+0x3a/0x8b
 [<ffffffff812fb3fa>] ? io_schedule+0x73/0xb7
 [<ffffffff8117f6e9>] ? get_request_wait+0xf0/0x188
 [<ffffffff81064f92>] ? autoremove_wake_function+0x0/0x2e
 [<ffffffff8117fa78>] ? __make_request+0x2f7/0x428
 [<ffffffff81191f63>] ? radix_tree_tag_clear+0x93/0xf1
 [<ffffffff8117e247>] ? generic_make_request+0x299/0x2f9
 [<ffffffff81192109>] ? radix_tree_tag_set+0x65/0xc6
 [<ffffffff8117e37d>] ? submit_bio+0xd6/0xf2
 [<ffffffff810ba9a9>] ? test_set_page_writeback+0xe0/0xef
 [<ffffffff810d8be8>] ? swap_writepage+0x9b/0xa5
 [<ffffffff810be529>] ? shrink_page_list+0x375/0x623
 [<ffffffff810bea8b>] ? shrink_active_list+0x2b4/0x2d9
 [<ffffffff810bef0c>] ? shrink_list+0x45c/0x767
 [<ffffffff812fb320>] ? thread_return+0x79/0xe0
 [<ffffffff810bf497>] ? shrink_zone+0x280/0x342
 [<ffffffff812fb4ab>] ? _cond_resched+0x24/0x2f
 [<ffffffff810bf69a>] ? shrink_slab+0x141/0x153
 [<ffffffff810c055e>] ? try_to_free_pages+0x232/0x38e
 [<ffffffff810bd553>] ? isolate_pages_global+0x0/0x20f
 [<ffffffff810ba5ec>] ? __alloc_pages_nodemask+0x3d4/0x5fc
 [<ffffffff810d8e00>] ? read_swap_cache_async+0x5d/0xf3
 [<ffffffff810d8eed>] ? swapin_readahead+0x57/0x98
 [<ffffffff810ccdfd>] ? handle_mm_fault+0x47f/0x80f
 [<ffffffff812feb96>] ? do_page_fault+0x2e0/0x2fc
 [<ffffffff812fca35>] ? page_fault+0x25/0x30

-- 
ciao,
Marco

Attachment: signature.asc
Description: Digital signature


Reply to: