Re: Kernel 2.6.32 "blocked for more than 120 seconds"
Same here. When a server is very busy I have the same message. In my
case it is a during a maasive update of a Postgres database.
Kernel: 2.6.32-bpo.3-686-bigmem
CPU: Intel(R) Xeon(R) CPU E5335 @ 2.00GHz
RAM; 4 Gb
[8567286.800035] INFO: task xfssyncd:1755 blocked for more than 120 seconds.
[8567286.821606] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[8567286.843156] xfssyncd D 00000010 0 1755 2 0x00000000
[8567286.843162] f68af700 00000046 00000000 00000010 05d5ca60 c1420d20
c1420d20 00000000
[8567286.864790] f68af8bc c4248d20 00000001 71cc5cc8 001e6fca c141da38
e1f0d380 00000010
[8567286.886304] 00000001 f68af8bc 00001000 00000000 e1f0d380 c1603720
00001000 000004c1
[8567286.907990] Call Trace:
[8567286.929085] [<c126e8d9>] ? schedule_timeout+0x20/0xb0
[8567286.950502] [<f87a5c22>] ? _xfs_buf_ioapply+0x206/0x22b [xfs]
[8567286.971982] [<c126e7e2>] ? wait_for_common+0xa5/0x101
[8567286.993546] [<c1033a25>] ? default_wake_function+0x0/0x8
[8567287.015186] [<f87ac6ec>] ? xfs_reclaim_inode+0xf4/0x117 [xfs]
[8567287.037271] [<f87ace2c>] ? xfs_inode_ag_walk+0x44/0x73 [xfs]
[8567287.059269] [<f87ac70f>] ? xfs_reclaim_inode_now+0x0/0x4c [xfs]
[8567287.081531] [<f87ace91>] ? xfs_inode_ag_iterator+0x36/0x58 [xfs]
[8567287.103816] [<f87ac70f>] ? xfs_reclaim_inode_now+0x0/0x4c [xfs]
[8567287.126334] [<f87acec1>] ? xfs_reclaim_inodes+0xe/0x10 [xfs]
[8567287.148852] [<f87acee6>] ? xfs_sync_worker+0x23/0x5c [xfs]
[8567287.171378] [<f87ac8f1>] ? xfssyncd+0x134/0x17d [xfs]
[8567287.193593] [<f87ac7bd>] ? xfssyncd+0x0/0x17d [xfs]
[8567287.215280] [<c104a660>] ? kthread+0x61/0x66
[8567287.236501] [<c104a5ff>] ? kthread+0x0/0x66
[8567287.257304] [<c1008bc7>] ? kernel_thread_helper+0x7/0x10
Op 7-9-2010 15:31, Erez Zarum schreef:
>
>
> I'm having this issue on all of my servers running the linux-image-amd64
> from the backports (2.6.32-bpo.5-amd64).
>
> I can't find a way to reproduce it, but i have one server that does a
> lot of fsync commands (mongodb) and i see if more often there as well.
>
> I also have a few servers with a stock kernel and they don't have those
> kind of errors at all.
>
> The reason i do use the kernel from backports is because i'm in need for
> the mpt2sas modules.
>
>
> [613954.220010] INFO: task mysqld:9935 blocked for more than 120 seconds.
>
> [613954.220044] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [613954.220092] mysqld D 0000000000000000 0 9935 9716
> 0x00000000
> [613954.220134] ffff88043e4de2e0 0000000000000086 0000000000000000
> ffffffff810ec6cd
> [613954.220185] ffff88043ad60900 000000000000f8a0 ffff88043cb43fd8
> 0000000000015640
> [613954.220235] 0000000000015640 ffff880242585bd0 ffff880242585ec8
> 0000000700000024
> [613954.220286] Call Trace:
> [613954.220311] [<ffffffff810ec6cd>] ? __dentry_open+0x1c4/0x2bf
> [613954.220341] [<ffffffff812fa62c>] ? rwsem_down_failed_common+0x8c/0xa8
> [613954.220371] [<ffffffff812fa664>] ? rwsem_down_write_failed+0x1c/0x25
> [613954.220402] [<ffffffff81194b23>] ?
> call_rwsem_down_write_failed+0x13/0x20
> [613954.220432] [<ffffffff812fa023>] ? down_write+0x25/0x27
> [613954.220460] [<ffffffff810c64f7>] ? sys_mmap_pgoff+0xd2/0x19d
> [613954.220490] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b
>
>
> [132588.251800] INFO: task mongod:4865 blocked for more than 120 seconds.
> [132588.258497] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [132588.266534] mongod D 0000000000000002 0 4865 1
> 0x00000000
> [132588.273659] ffff88043e62aa60 0000000000000086 ffff88043aa61a70
> ffffffff8117d4dd
> [132588.286788] ffff880439d21800 000000000000f9e0 ffff88043c4e5fd8
> 0000000000015780
> [132588.294402] 0000000000015780 ffff88043b59b880 ffff88043b59bb78
> 00000004370a1100
> [132588.302135] Call Trace:
> [132588.304729] [<ffffffff8117d4dd>] ? blk_plug_device+0x5a/0x82
> [132588.310641] [<ffffffff81016539>] ? read_tsc+0xa/0x20
> [132588.315882] [<ffffffff810b2865>] ? sync_page+0x0/0x46
> [132588.321216] [<ffffffff812f7ae8>] ? io_schedule+0x73/0xb7
> [132588.326852] [<ffffffff810b28a6>] ? sync_page+0x41/0x46
> [132588.332226] [<ffffffff812f7ff5>] ? __wait_on_bit+0x41/0x70
> [132588.337968] [<ffffffff810b2a2a>] ? wait_on_page_bit+0x6b/0x71
> [132588.344009] [<ffffffff810638c8>] ? wake_bit_function+0x0/0x23
> [132588.350073] [<ffffffff810ba65e>] ? pagevec_lookup_tag+0x1a/0x21
> [132588.356297] [<ffffffff810b31ff>] ?
> wait_on_page_writeback_range+0x69/0x11b
> [132588.363494] [<ffffffff810b3325>] ?
> __filemap_fdatawrite_range+0x4b/0x54
> [132588.370539] [<ffffffff810b3371>] ?
> filemap_write_and_wait_range+0x43/0x52
> [132588.377810] [<ffffffff81109df6>] ? vfs_fsync_range+0x55/0x9e
> [132588.383756] [<ffffffff810d2270>] ? sys_msync+0x110/0x180
> [132588.389505] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b
>
>
> Erez.
>
>
--
Met vriendelijke groet/Regards
Prompt
R.J. Baart
Kerkstraat 173
5261CW Vught
tel: +31 73 6567041
mailto:R.J.Baart@Prompt.NL
Public PGP key: subkeys.pgp.net
Reply to: