Bug#534880: linux-image-2.6.26-2-xen-686: domU hang and are unresponsive
Same thing happening here every few days. I managed to get some more info
using crash utility and a core generated with xm dump-core. To get a
vmlinux with debug symbols I recompiled the linux-image-2.6.26-2-xen
package with CONFIG_DEBUG_INFO.
The strange thing is that it seems to be stuck on the first instruction
of thread_return on all three virtual CPUs. Perhaps someone else can make
a core dump and check if it hangs in the same place.
$ ./crash /boot/System.map-2.6.26-2-xen-amd64 ./vmlinux node.core
crash 4.0.9
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
SYSTEM MAP: /boot/System.map-2.6.26-2-xen-amd64
DEBUG KERNEL: ./vmlinux (2.6.26-2-xen-amd64)
DUMPFILE: node.core
CPUS: 3
DATE: Wed Sep 30 09:38:19 2009
UPTIME: 24 days, 20:48:52
LOAD AVERAGE: 0.16, 0.11, 0.09
TASKS: 106
NODENAME: node
RELEASE: 2.6.26-2-xen-amd64
VERSION: #1 SMP Thu Aug 20 00:36:34 UTC 2009
MACHINE: x86_64 (2333 Mhz)
MEMORY: 3 GB
PANIC: ""
PID: 0
COMMAND: "swapper"
TASK: ffffffff804ff460 (1 of 3) [THREAD_INFO: ffffffff80552000]
CPU: 0
STATE: TASK_RUNNING
WARNING: panic task not found
crash> bt -a -t
PID: 23666 TASK: ffff88005df662c0 CPU: 0 COMMAND: "apache2"
START: thread_return (schedule) at ffffffff80434c7d
[ffff88005f1f5eb0] touch_atime at ffffffff8029e525
[ffff88005f1f5ef0] vfs_readdir at ffffffff80296c35
[ffff88005f1f5f30] sys_getdents at ffffffff80296e70
[ffff88005f1f5f80] system_call at ffffffff8020b528
[ffff88005f1f5fb0] system_call at ffffffff8020b4c0
RIP: 00007f27928fd1a5 RSP: 00007fff9b663060 RFLAGS: 00000246
RAX: 000000000000004e RBX: ffffffff8020b528 RCX: ffffffff8020b4c0
RDX: 0000000000001000 RSI: 0000000003a3de28 RDI: 000000000000000d
RBP: 0000000003a3de28 R8: 0000000003a3de28 R9: 0000000000000002
R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000001000
R13: 0000000003a3de00 R14: 00007fff9b664250 R15: 0000000000000008
ORIG_RAX: 000000000000004e CS: e033 SS: e02b
PID: 0 TASK: ffff88005fc5e6c0 CPU: 1 COMMAND: "swapper"
START: thread_return (schedule) at ffffffff80434c7d
[ffff88005fc67eb0] hypercall_page at ffffffff802083aa
[ffff88005fc67ed8] hypercall_page at ffffffff802083aa
[ffff88005fc67f18] xen_safe_halt at ffffffff8020e79d
[ffff88005fc67f38] xen_idle at ffffffff8020a0c8
[ffff88005fc67f48] cpu_idle at ffffffff80209cd6
PID: 0 TASK: ffff88005fc5e040 CPU: 2 COMMAND: "swapper"
START: thread_return (schedule) at ffffffff80434c7d
[ffff88005fc69eb0] hypercall_page at ffffffff802083aa
[ffff88005fc69ed8] hypercall_page at ffffffff802083aa
[ffff88005fc69f18] xen_safe_halt at ffffffff8020e79d
[ffff88005fc69f38] xen_idle at ffffffff8020a0c8
[ffff88005fc69f48] cpu_idle at ffffffff80209cd6
crash> dis 0xffffffff80434c7d
0xffffffff80434c7d <thread_return>: mov %gs:0x0,%rsi
crash> kmem -i
PAGES TOTAL PERCENTAGE
TOTAL MEM 393270 1.5 GB ----
FREE 57786 225.7 MB 14% of TOTAL MEM
USED 335484 1.3 GB 85% of TOTAL MEM
SHARED 46295 180.8 MB 11% of TOTAL MEM
BUFFERS 11449 44.7 MB 2% of TOTAL MEM
CACHED 208033 812.6 MB 52% of TOTAL MEM
SLAB 18242 71.3 MB 4% of TOTAL MEM
TOTAL SWAP 0 0 ----
SWAP USED 0 0 100% of TOTAL SWAP
SWAP FREE 0 0 0% of TOTAL SWAP
--
Valentin
Reply to: