I experienced this problem on a machine with 12gigs of memory when I added a failed disk back into a mdadm array, output included below. The machine basically was totally frozen for 10 minutes or so until the raid array finished its sync, and then everything returned to normal. Pretty ugly, would love to see this merged in to a lenny kernel. I got pages and pages of the following, so I'm only showing the end: [1540865.693902] INFO: task sshd:15082 blocked for more than 120 seconds. [1540865.693956] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [1540865.694002] sshd D ed0b2780 0 15082 23253 [1540865.694077] ec4450c0 00200282 c018d895 ed0b2780 c01b7f2e ec445248 c153fb40 00000003 [1540865.694213] ec5cb080 ec4450c0 16ed73c7 ed43eacc ec44530c ffffffff eccad200 c1679020 [1540865.694361] ed1cc0d4 c018d8c6 00002c2c 00000000 00000000 00000000 e7f11d8c c7fb7dbc [1540865.694509] Call Trace: [1540865.694576] [<c018d895>] __find_get_block+0x198/0x1a2 [1540865.694636] [<c01b7f2e>] security_inode_alloc+0x16/0x17 [1540865.694704] [<c018d8c6>] __getblk+0x27/0x26f [1540865.694757] [<ee1ef6a0>] do_get_write_access+0x1b6/0x367 [jbd] [1540865.694829] [<c012ec51>] wake_bit_function+0x0/0x3c [1540865.694882] [<ee1ef869>] journal_get_write_access+0x18/0x26 [jbd] [1540865.694942] [<ee22e5df>] __ext3_journal_get_write_access+0x13/0x32 [ext3] [1540865.695012] [<ee2237e2>] ext3_reserve_inode_write+0x2d/0x5d [ext3] [1540865.695067] [<ee223823>] ext3_mark_inode_dirty+0x11/0x27 [ext3] [1540865.695132] [<ee225fcc>] ext3_dirty_inode+0x50/0x63 [ext3] [1540865.695194] [<c018a3f5>] __mark_inode_dirty+0x21/0x148 [1540865.695245] [<c0181c29>] touch_atime+0xc7/0xce [1540865.695294] [<c015022c>] generic_file_aio_read+0x493/0x4da [1540865.695370] [<c016fd62>] do_sync_read+0xbf/0xfe [1540865.695436] [<c012ec24>] autoremove_wake_function+0x0/0x2d [1540865.695495] [<ee230a51>] ext3_xattr_security_get+0x31/0x3b [ext3] [1540865.695562] [<c01b7f6f>] security_file_permission+0xc/0xd [1540865.695621] [<c016fca3>] do_sync_read+0x0/0xfe [1540865.695674] [<c01704ea>] vfs_read+0x81/0x11e [1540865.695724] [<c01738d4>] kernel_read+0x32/0x43 [1540865.695775] [<c01739a5>] prepare_binprm+0xc0/0xc4 [1540865.695827] [<c0174960>] do_execve+0xd2/0x1c6 [1540865.695879] [<c0102366>] sys_execve+0x2a/0x4a [1540865.695926] [<c0103f76>] syscall_call+0x7/0xb [1540865.695988] [<c02c0000>] pci_bus_size_bridges+0x32a/0x3a0 [1540865.696044] ======================= [1540869.985790] INFO: task sshd:15078 blocked for more than 120 seconds. [1540869.985845] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [1540869.988032] sshd D ed0b2780 0 15078 23253 [1540869.988110] ec4af140 00200286 c018d895 ed0b2780 c01b7f2e ec4af2c8 c153fb40 00000003 [1540869.988236] eca48300 ec4af140 16ecfe8f ed43eacc ec4af38c ffffffff eccad200 c1679020 [1540869.988379] ed1cc0d4 c018d8c6 00002929 00000000 00000000 00000000 ec38dd8c c7fb7dbc [1540869.988529] Call Trace: [1540869.988595] [<c018d895>] __find_get_block+0x198/0x1a2 [1540869.988653] [<c01b7f2e>] security_inode_alloc+0x16/0x17 [1540869.988720] [<c018d8c6>] __getblk+0x27/0x26f [1540869.988772] [<ee1ef6a0>] do_get_write_access+0x1b6/0x367 [jbd] [1540869.988847] [<c012ec51>] wake_bit_function+0x0/0x3c [1540869.988899] [<ee1ef869>] journal_get_write_access+0x18/0x26 [jbd] [1540869.988960] [<ee22e5df>] __ext3_journal_get_write_access+0x13/0x32 [ext3] [1540869.989033] [<ee2237e2>] ext3_reserve_inode_write+0x2d/0x5d [ext3] [1540869.989093] [<ee223823>] ext3_mark_inode_dirty+0x11/0x27 [ext3] [1540869.989159] [<ee225fcc>] ext3_dirty_inode+0x50/0x63 [ext3] [1540869.989218] [<c018a3f5>] __mark_inode_dirty+0x21/0x148 [1540869.989276] [<c0181c29>] touch_atime+0xc7/0xce [1540869.989326] [<c015022c>] generic_file_aio_read+0x493/0x4da [1540869.989384] [<c016fd62>] do_sync_read+0xbf/0xfe [1540869.989384] [<c012ec24>] autoremove_wake_function+0x0/0x2d [1540869.989384] [<ee230a51>] ext3_xattr_security_get+0x31/0x3b [ext3] [1540869.989384] [<c01b7f6f>] security_file_permission+0xc/0xd [1540869.989384] [<c016fca3>] do_sync_read+0x0/0xfe [1540869.989384] [<c01704ea>] vfs_read+0x81/0x11e [1540869.989384] [<c01738d4>] kernel_read+0x32/0x43 [1540869.989384] [<c01739a5>] prepare_binprm+0xc0/0xc4 [1540869.989384] [<c0174960>] do_execve+0xd2/0x1c6 [1540869.989384] [<c0102366>] sys_execve+0x2a/0x4a [1540869.989384] [<c0103f76>] syscall_call+0x7/0xb [1540869.989384] [<c02c0000>] pci_bus_size_bridges+0x32a/0x3a0 [1540869.989384] ======================= [1540874.198475] md: md2: recovery done. Login timed out after 60 seconds. [1540874.405091] RAID1 conf printout: [1540874.405173] --- wd:2 rd:3 [1540874.405210] disk 0, wo:0, o:1, dev:sdb4 [1540874.405260] disk 1, wo:0, o:1, dev:sdc4 Debian GNU/Linux 5.0 waxwing hvc0
Attachment:
signature.asc
Description: Digital signature