I too have been a victim of this problem, on a number of machines. I fed the lines into addr2line -e vmlinux after rebuilding the kernel with this flavor to get the vmlinux. This looks clearly like a xen bug, the kernel does some I/O, which gets queued, but for some reason the request causes a violation in xen. I also looked up the other crashes that people have reported here, they all seem to be either irq or scheduling related (involving the serial_core, or ipmi in those cases) and all seem to reference the following common point: /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/arch/x86/kernel/head_32-xen.S:72 Please see below for the specific line numbers associated with this. First the crash: [ 6930.059040] iret exception: 0000 [#1] SMP [ 6930.059123] Modules linked in: xt_tcpudp xt_physdev iptable_filter ip_tables x_tables netloop tun bridge ipv6 loop evdev container button i2c_i801 i2c_core intel_rng rng_core pcspkr shpchp pci_hotplug e7xxx_edac edac_core ext3 jbd mbcache serpent xts gf128mul dm_crypt crypto_blkcipher dm_mirror dm_log dm_snapshot dm_mod raid1 md_mod ide_disk ide_cd_mod cdrom ata_generic libata dock ide_pci_generic floppy aic79xx scsi_transport_spi scsi_mod piix ide_core e1000 uhci_hcd usbcore thermal processor fan thermal_sys [ 6930.060105] [ 6930.060146] Pid: 1241, comm: md1_resync Not tainted (2.6.26-1-xen-686 #1) [ 6930.060196] EIP: 0061:[<c01011a7>] EFLAGS: 00000002 CPU: 0 [ 6930.060246] EIP is at 0xc01011a7 [ 6930.060283] EAX: 00000000 EBX: ecfa3df0 ECX: 00000001 EDX: 00000001 [ 6930.060331] ESI: ecfa3df0 EDI: c1118e20 EBP: ece26040 ESP: ecfa3db4 [ 6930.060378] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069 [ 6930.060420] Process md1_resync (pid: 1241, ti=ecfa2000 task=ece26040 task.ti=ec806000) [ 6930.060472] Stack: c0102eb7 c1118e20 ece26040 ece2628c c111906c 00000000 00000001 ed4cac84 [ 6930.060501] ecd98ef0 ed4cac80 ebc08cac 00000003 c01db664 ebc08cac ecfa3e10 00000003 [ 6930.060501] 00000000 00000068 ecfa3ff8 db744000 00000002 ecfa8f00 c10c2038 00001000 [ 6930.060501] Call Trace: [ 6930.060501] [<c0102eb7>] __switch_to+0x376/0x42e [ 6930.060501] [<c01db664>] cfq_add_rq_rb+0x5c/0x6b [ 6930.060501] [<c01cf34e>] elv_merged_request+0x28/0x30 [ 6930.060501] [<c01d2140>] __make_request+0x294/0x36e [ 6930.060502] [<c01d0eb1>] generic_make_request+0x34d/0x37b [ 6930.060502] [<c0105f7c>] get_nsec_offset+0xe/0x6a [ 6930.060573] [<c0106151>] get_runstate_snapshot+0x69/0xec [ 6930.060573] [<c0115353>] update_curr+0x47/0x79 [ 6930.060573] [<c01160a3>] dequeue_entity+0x13/0x9b [ 6930.060573] [<c0115fa3>] __dequeue_entity+0x1f/0x71 [ 6930.060573] [<c02ca3a2>] schedule+0x616/0x6b1 [ 6930.060573] [<c0105b0d>] do_IRQ+0x52/0x65 [ 6930.060573] [<c02ca5dd>] schedule_timeout+0x13/0x86 [ 6930.060573] [<c012ed33>] prepare_to_wait+0x12/0x49 [ 6930.060573] [<ee1a6411>] md_thread+0x9c/0xcd [md_mod] [ 6930.060573] [<c012ec28>] autoremove_wake_function+0x0/0x2d [ 6930.060573] [<ee1a6375>] md_thread+0x0/0xcd [md_mod] [ 6930.060573] [<c012eb65>] kthread+0x38/0x5f [ 6930.060573] [<c012eb2d>] kthread+0x0/0x5f [ 6930.060573] [<c0104267>] kernel_thread_helper+0x7/0x10 [ 6930.060573] ======================= [ 6930.060573] Code: cc cc cc cc b8 0c 00 00 00 cd 82 c3 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc b8 0d 00 00 00 cd 82 <c3> cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc [ 6930.060573] EIP: [<c01011a7>] 0xc01011a7 SS:ESP 0069:ecfa3db4 Then, I fed each of the [<xxxx>] numbers through addr2line, including the EIP, which gives us: lenny:~/kernel/linux-2.6-2.6.26# addr2line -e ./debian/build/build_i386_xen_686/vmlinux c0102eb7 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/include/xen/hypercall.h:13 c01db664 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/block/cfq-iosched.c:1464 c01cf34e /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/block/elevator.c:118 c01d2140 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/block/blk-core.c:413 c01d0eb1 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/block/blk-core.c:1228 c0105f7c /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/arch/x86/kernel/time_32-xen.c:235 c0106151 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/arch/x86/kernel/time_32-xen.c:206 c0115353 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/kernel/sched_fair.c:433 c01160a3 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/kernel/sched_fair.c:738 c0115fa3 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/kernel/sched.c:1260 c02ca3a2 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/kernel/sched.c:3435 c0105b0d /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/include/asm/irq_regs_32.h:24 c02ca5dd /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/kernel/sched_stats.h:195 c012ed33 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/kernel/wait.c:125 ee1a6411 ??:0 c012ec28 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/kernel/wait.c:248 ee1a6375 ??:0 c012eb65 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/kernel/kthread.c:150 c012eb2d /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/include/linux/list.h:45 c0104267 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/arch/x86/kernel/entry_32-xen.S:1259 c01011a7 /root/kernel/linux-2.6-2.6.26/debian/build/build_i386_xen_686/arch/x86/kernel/head_32-xen.S:72 micah
Attachment:
signature.asc
Description: Digital signature