[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#516374: Me too, and it just got worse



I'm also having this problem.

Dom0 running 2.6.26-2-xen-amd64, domU on 2.6.26-2-686-bigmem. All domU:s are using all available processor cores.

I've been having problems on and off since the machine was installed last summer. Typically it would be days or weeks between lockups.

I've been keeping up with the latest stable kernel version, but so far the upgrades haven't made any difference.

The lastest upgrade (2.6.26-21lenny4) unfortunately made things worse. Now one of my domUs lockup in less than an hour.

I get two different error messages in kern.log. Lots of "task xx blocked for more than 120 seconds" and fewer "BUG: soft lockup - CPU#n stuck...".

Once a domU is stuck, there is no way to reboot it other than using xm/virsh destroy.

Here are examples of the two types of output in the log:

INFO: task nfsd:1700 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
nfsd          D 7caf7bc7     0  1700      2
ec94ce60 00000246 00000000 7caf7bc7 00001c5b ec94cfec c2578020 00000000 c0130978 ebb87e64 0000a760 00000000 c041a1bc c0130a8b 00000000 00000200 0086759a 0086759a ec528600 e4c3675c c02c9057 eb8f3e64 c041b220 0086759a
Call Trace:
 [<c0130978>] lock_timer_base+0x19/0x35
 [<c0130a8b>] __mod_timer+0x99/0xa3
 [<c02c9057>] schedule_timeout+0x6b/0x86
 [<c01306b4>] process_timeout+0x0/0x5
 [<c02c9052>] schedule_timeout+0x66/0x86
 [<ee0413a0>] journal_stop+0x7e/0x151 [jbd]
 [<c0196930>] __writeback_single_inode+0x15a/0x251
 [<c0196a8a>] write_inode_now+0x63/0x9a
 [<ee465227>] nfsd_setattr+0x3ae/0x3cb [nfsd]
 [<ee46ac81>] nfsd3_proc_setattr+0x74/0x7d [nfsd]
 [<ee461205>] nfsd_dispatch+0xca/0x192 [nfsd]
 [<ee3c1fad>] svc_process+0x3a1/0x620 [sunrpc]
 [<ee461731>] nfsd+0x171/0x268 [nfsd]
 [<ee4615c0>] nfsd+0x0/0x268 [nfsd]
 [<c01094f7>] kernel_thread_helper+0x7/0x10
 =======================


BUG: soft lockup - CPU#3 stuck for 71s! [swapper:0]
Modules linked in: autofs4 nfsd auth_rpcgss exportfs nfs lockd nfs_acl sunrpc ipv6 nf_conntrack_ipv4 xt_state nf_conntrack xt_limit ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables loop evdev xen_netfront pcspkr ext3 jbd mbcache xen_blkfront thermal_sys

Pid: 0, comm: swapper Not tainted (2.6.26-2-686-bigmem #1)
EIP: 0061:[<c01023a7>] EFLAGS: 00000246 CPU: 3
EIP is at _stext+0x3a7/0x1000
EAX: 00000000 EBX: 00000001 ECX: 00000000 EDX: 00867599
ESI: 00000003 EDI: 00000000 EBP: 00000000 ESP: ed049fa0
 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
CR0: 8005003b CR2: b620d034 CR3: 2c1a0000 CR4: 00000660
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400
 [<c0103da5>] xen_safe_halt+0xd/0x17
 [<c0104c0a>] xen_idle+0x0/0x3a
 [<c0104c35>] xen_idle+0x2b/0x3a
 [<c01075d3>] cpu_idle+0xb0/0xd0
 =======================





Reply to: