Kernel 2.6.32 "blocked for more than 120 seconds"
I'm having this issue on all of my servers running the linux-image-amd64
from the backports (2.6.32-bpo.5-amd64).
I can't find a way to reproduce it, but i have one server that does a
lot of fsync commands (mongodb) and i see if more often there as well.
I also have a few servers with a stock kernel and they don't have those
kind of errors at all.
The reason i do use the kernel from backports is because i'm in need for
the mpt2sas modules.
[613954.220010] INFO: task mysqld:9935 blocked for more than 120 seconds.
[613954.220044] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[613954.220092] mysqld D 0000000000000000 0 9935 9716
0x00000000
[613954.220134] ffff88043e4de2e0 0000000000000086 0000000000000000
ffffffff810ec6cd
[613954.220185] ffff88043ad60900 000000000000f8a0 ffff88043cb43fd8
0000000000015640
[613954.220235] 0000000000015640 ffff880242585bd0 ffff880242585ec8
0000000700000024
[613954.220286] Call Trace:
[613954.220311] [<ffffffff810ec6cd>] ? __dentry_open+0x1c4/0x2bf
[613954.220341] [<ffffffff812fa62c>] ? rwsem_down_failed_common+0x8c/0xa8
[613954.220371] [<ffffffff812fa664>] ? rwsem_down_write_failed+0x1c/0x25
[613954.220402] [<ffffffff81194b23>] ?
call_rwsem_down_write_failed+0x13/0x20
[613954.220432] [<ffffffff812fa023>] ? down_write+0x25/0x27
[613954.220460] [<ffffffff810c64f7>] ? sys_mmap_pgoff+0xd2/0x19d
[613954.220490] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b
[132588.251800] INFO: task mongod:4865 blocked for more than 120 seconds.
[132588.258497] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[132588.266534] mongod D 0000000000000002 0 4865 1
0x00000000
[132588.273659] ffff88043e62aa60 0000000000000086 ffff88043aa61a70
ffffffff8117d4dd
[132588.286788] ffff880439d21800 000000000000f9e0 ffff88043c4e5fd8
0000000000015780
[132588.294402] 0000000000015780 ffff88043b59b880 ffff88043b59bb78
00000004370a1100
[132588.302135] Call Trace:
[132588.304729] [<ffffffff8117d4dd>] ? blk_plug_device+0x5a/0x82
[132588.310641] [<ffffffff81016539>] ? read_tsc+0xa/0x20
[132588.315882] [<ffffffff810b2865>] ? sync_page+0x0/0x46
[132588.321216] [<ffffffff812f7ae8>] ? io_schedule+0x73/0xb7
[132588.326852] [<ffffffff810b28a6>] ? sync_page+0x41/0x46
[132588.332226] [<ffffffff812f7ff5>] ? __wait_on_bit+0x41/0x70
[132588.337968] [<ffffffff810b2a2a>] ? wait_on_page_bit+0x6b/0x71
[132588.344009] [<ffffffff810638c8>] ? wake_bit_function+0x0/0x23
[132588.350073] [<ffffffff810ba65e>] ? pagevec_lookup_tag+0x1a/0x21
[132588.356297] [<ffffffff810b31ff>] ?
wait_on_page_writeback_range+0x69/0x11b
[132588.363494] [<ffffffff810b3325>] ? __filemap_fdatawrite_range+0x4b/0x54
[132588.370539] [<ffffffff810b3371>] ?
filemap_write_and_wait_range+0x43/0x52
[132588.377810] [<ffffffff81109df6>] ? vfs_fsync_range+0x55/0x9e
[132588.383756] [<ffffffff810d2270>] ? sys_msync+0x110/0x180
[132588.389505] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b
Erez.
Reply to: