Bug#557763: linux-image-2.6.26-2-686: crashes intermittedly under kvm
Package: linux-image-2.6.26-2-686
Version: 2.6.26-19lenny2
Hi,
I am, on multiple VMs seeing crashes at the rate of approximately once a
week. All of the crashes are in skb_release_all+0x48/0xa0.
I managed to get a backtrace:
Nov 24 03:55:05 bhuna [233316.039474] BUG: unable to handle kernel
Nov 24 03:55:05 bhuna [233316.039474] WARNING: at net/core/skbuff.c:368 skb_release_all+0x48/0xa0()
Nov 24 03:55:05 bhuna [233316.039474] Modules linked in: ipv6 quota_v2 netconsole configfs virtio_net snd_pcm snd_timer snd soundcore snd_page_al
loc psmouse pcspkr serio_raw button i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_log dm_snapshot dm_mod raid10 raid456 async_xor async_
memcpy async_tx xor raid1 raid0 multipath linear md_mod virtio_blk ide_cd_mod cdrom ata_generic libata scsi_mod dock floppy uhci_hcd virtio_pci p
iix ide_pci_generic usbcore ide_core thermal processor fan thermal_sys [last unloaded: scsi_wait_scan]
Nov 24 03:55:05 bhuna [233316.039474] Pid: 28778, comm: imap Tainted: G S 2.6.26-2-686 #1
Nov 24 03:55:05 bhuna [233316.039474] [<c01225f3>] warn_on_slowpath+0x40/0x66
Nov 24 03:55:05 bhuna [233316.039474] [<c01e1699>] vsnprintf+0x411/0x452
Nov 24 03:55:05 bhuna [233316.039474] [<c01319a9>] autoremove_wake_function+0xd/0x2d
Nov 24 03:55:05 bhuna [233316.039474] [<c01184d1>] __wake_up_common+0x2e/0x58
Nov 24 03:55:05 bhuna [233316.039474] [<c011a6bb>] __wake_up+0x29/0x39
Nov 24 03:55:05 bhuna [233316.039474] [<c0254922>] skb_release_all+0x48/0xa0
Nov 24 03:55:05 bhuna [233316.039474] [<c0254287>] __kfree_skb+0x8/0x61
Nov 24 03:55:05 bhuna [233316.039474] [<f896703b>] free_old_xmit_skbs+0x3b/0x4f [virtio_net]
Nov 24 03:55:05 bhuna [233316.039474] [<f8967439>] start_xmit+0x14/0xa5 [virtio_net]
Nov 24 03:55:05 bhuna [233316.039474] [<c0263730>] netpoll_send_skb+0xba/0x156
Nov 24 03:55:05 bhuna [233316.039474] [<f897b0b2>] write_msg+0x6d/0xa9 [netconsole]
Nov 24 03:55:05 bhuna [233316.039474] [<f897b045>] write_msg+0x0/0xa9 [netconsole]
Nov 24 03:55:05 bhuna [233316.039474] [<c012266e>] __call_console_drivers+0x4f/0x5b
Nov 24 03:55:05 bhuna [233316.039474] [<c0122a61>] release_console_sem+0x140/0x18c
Nov 24 03:55:05 bhuna [233316.039474] [<c0122ed6>] vprintk+0x29d/0x2de
Nov 24 03:55:05 bhuna [233316.039474] [<c0122f2b>] printk+0x14/0x18
Nov 24 03:55:05 bhuna [233316.039474] [<c0115d21>] do_page_fault+0x435/0x5c0
Nov 24 03:55:05 bhuna [233316.039474] [<c01e1699>] vsnprintf+0x411/0x452
Nov 24 03:55:05 bhuna [233316.039474] [<c01158ec>] do_page_fault+0x0/0x5c0
Nov 24 03:55:05 bhuna [233316.039474] [<c02b9de2>] error_code+0x72/0x78
Nov 24 03:55:05 bhuna [233316.039474] [<c016a080>] __bounce_end_io_read+0x88/0xd7
Nov 24 03:55:06 bhuna [233316.039474] [<c016a0dc>] bounce_end_io_read+0x0/0xd
Nov 24 03:55:06 bhuna [233316.039474] [<c0192d38>] bio_endio+0x26/0x28
Nov 24 03:55:06 bhuna [233316.039474] [<c01d1de3>] __end_that_request_first+0x1b2/0x2c3
Nov 24 03:55:06 bhuna [233316.039474] [<c01d2712>] __blk_end_request+0x16/0x37
Nov 24 03:55:06 bhuna [233316.039474] [<f880a293>] blk_done+0x3d/0x88 [virtio_blk]
Nov 24 03:55:06 bhuna [233316.039474] [<c024c6cb>] vring_interrupt+0x23/0x2e
Nov 24 03:55:06 bhuna [233316.039474] [<f882f0a6>] vp_interrupt+0x67/0xa4 [virtio_pci]
Nov 24 03:55:06 bhuna [233316.039474] [<c01520ea>] handle_IRQ_event+0x23/0x51
Nov 24 03:55:06 bhuna [233316.039474] [<c01531e9>] handle_fasteoi_irq+0x71/0xa4
Nov 24 03:55:06 bhuna [233316.039474] [<c0105f3a>] do_IRQ+0x4d/0x63
Nov 24 03:55:06 bhuna [233316.039474] [<c01042a7>] common_interrupt+0x23/0x28
Nov 24 03:55:06 bhuna [233316.039474] [<c01265e5>] __do_softirq+0x56/0xd3
Nov 24 03:55:06 bhuna [233316.039474] [<c01266a7>] do_softirq+0x45/0x53
Nov 24 03:55:06 bhuna [233316.039474] [<c012695e>] irq_exit+0x35/0x67
Nov 24 03:55:06 bhuna [233316.039474] [<c01101c9>] smp_apic_timer_interrupt+0x6b/0x76
Nov 24 03:55:06 bhuna [233316.039474] [<c0104364>] apic_timer_interrupt+0x28/0x30
Nov 24 03:55:06 bhuna [233316.039474] [<c0115009>] paravirt_leave_lazy+0x1c/0x1d
Nov 24 03:55:06 bhuna [233316.039474] [<c0114b44>] kvm_leave_lazy_mmu+0x24/0x31
Nov 24 03:55:06 bhuna [233316.039474] [<c0162a80>] unmap_vmas+0x3f0/0x5ad
Nov 24 03:55:06 bhuna [233316.039474] [<c01dffd0>] rb_insert_color+0x4c/0xad
Nov 24 03:55:06 bhuna [233316.039474] [<c0165fbb>] vma_adjust+0x310/0x399
Nov 24 03:55:06 bhuna [233316.039474] [<c016587d>] unmap_region+0x83/0xe9
Nov 24 03:55:06 bhuna [233316.039474] [<c016625f>] do_munmap+0x148/0x19b
Nov 24 03:55:06 bhuna [233316.039474] [<c01668cf>] mmap_region+0x6f/0x393
Nov 24 03:55:06 bhuna [233316.039474] [<c0166f87>] arch_get_unmapped_area_topdown+0x0/0x120
Nov 24 03:55:07 bhuna [233316.039474] [<c0166f34>] do_mmap_pgoff+0x266/0x2b9
Nov 24 03:55:07 bhuna [233316.039474] [<c0106a4e>] sys_mmap2+0x86/0xa0
Nov 24 03:55:07 bhuna [233316.039474] [<c01038ce>] syscall_call+0x7/0xb
Nov 24 03:55:07 bhuna [233316.039474] =======================
Nov 24 03:55:07 bhuna [233316.039474] ---[ end trace 008841ced08c0d44 ]---
Nov 24 03:55:07 bhuna [233316.039474] paging request
Nov 24 03:55:07 at fffba000
Nov 24 03:55:07 bhuna [233316.039474] IP:
Nov 24 03:55:07 [<c016a080>] __bounce_end_io_read+0x88/0xd7
Nov 24 03:55:07 bhuna [233316.039475] *pde = 00007067
Nov 24 03:55:07 *pte = 00000000
Nov 24 03:55:07 bhuna
Nov 24 03:55:07 bhuna [233316.039475] Oops: 0002 [#1]
Nov 24 03:55:07 bhuna SMP
Nov 24 03:55:07 bhuna
Nov 24 03:55:07 bhuna [233316.039475] Modules linked in:
Nov 24 03:55:07 ipv6
Nov 24 03:55:07 quota_v2
Nov 24 03:55:07 netconsole
Nov 24 03:55:07 configfs
Nov 24 03:55:07 virtio_net
Nov 24 03:55:07 snd_pcm
Nov 24 03:55:07 snd_timer
Nov 24 03:55:07 snd
Nov 24 03:55:07 soundcore
Nov 24 03:55:07 snd_page_alloc
Nov 24 03:55:07 psmouse
Nov 24 03:55:07 pcspkr
Nov 24 03:55:07 serio_raw
Nov 24 03:55:07 button
Nov 24 03:55:07 i2c_piix4
Nov 24 03:55:07 i2c_core
Nov 24 03:55:07 evdev
Nov 24 03:55:07 ext3
Nov 24 03:55:07 jbd
Nov 24 03:55:07 mbcache
Nov 24 03:55:07 dm_mirror
Nov 24 03:55:07 dm_log
Nov 24 03:55:07 dm_snapshot
Nov 24 03:55:07 dm_mod
Nov 24 03:55:07 raid10
Nov 24 03:55:07 raid456
Nov 24 03:55:07 async_xor
Nov 24 03:55:07 async_memcpy
Nov 24 03:55:07 async_tx
Nov 24 03:55:07 xor
Nov 24 03:55:07 raid1
Nov 24 03:55:07 raid0
Nov 24 03:55:07 multipath
Nov 24 03:55:07 linear
Nov 24 03:55:07 md_mod
Nov 24 03:55:07 virtio_blk
Nov 24 03:55:07 ide_cd_mod
Nov 24 03:55:07 cdrom
Nov 24 03:55:07 ata_generic
Nov 24 03:55:07 libata
Nov 24 03:55:07 scsi_mod
Nov 24 03:55:07 dock
Nov 24 03:55:07 floppy
Nov 24 03:55:07 uhci_hcd
Nov 24 03:55:07 virtio_pci
Nov 24 03:55:07 piix
Nov 24 03:55:07 ide_pci_generic
Nov 24 03:55:07 usbcore
Nov 24 03:55:07 ide_core
Nov 24 03:55:07 thermal
Nov 24 03:55:07 processor
Nov 24 03:55:07 fan
Nov 24 03:55:07 thermal_sys
Nov 24 03:55:07 [last unloaded: scsi_wait_scan]
Nov 24 03:55:07 bhuna
Nov 24 03:55:07 bhuna [233316.039475]
Nov 24 03:55:07 bhuna [233316.039475] Pid: 28778, comm: imap Tainted: G S W (2.6.26-2-686 #1)
Nov 24 03:55:07 bhuna [233316.039475] EIP: 0060:[<c016a080>] EFLAGS: 00010086 CPU: 0
Nov 24 03:55:07 bhuna [233316.039475] EIP is at __bounce_end_io_read+0x88/0xd7
Nov 24 03:55:07 bhuna [233316.039475] EAX: fffba000 EBX: 00000086 ECX: 00000400 EDX: 00001000
Nov 24 03:55:07 bhuna [233316.039475] ESI: dcc5e000 EDI: fffba000 EBP: f74a034c ESP: c1c91cd4
Nov 24 03:55:07 bhuna [233316.039475] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Nov 24 03:55:07 bhuna [233316.039475] Process imap (pid: 28778, ti=c1c90000 task=c1cd75e0 task.ti=c1c90000)
Nov 24 03:55:07 bhuna
Nov 24 03:55:07 bhuna [233316.039475] Stack:
Nov 24 03:55:07 bhuna 00000000
Nov 24 03:55:07 bhuna 00000000
Nov 24 03:55:07 bhuna f74dbc00
Nov 24 03:55:07 bhuna decea640
Nov 24 03:55:07 bhuna dcd1eac0
Nov 24 03:55:07 bhuna 00000000
Nov 24 03:55:07 bhuna c016a0dc
Nov 24 03:55:07 bhuna decea640
Nov 24 03:55:07 bhuna
Nov 24 03:55:07 bhuna [233316.039475]
Nov 24 03:55:07 bhuna e27dad70
Nov 24 03:55:07 bhuna 00000000
Nov 24 03:55:07 bhuna c0192d38
Nov 24 03:55:07 bhuna 00004000
Nov 24 03:55:07 bhuna c01d1de3
Nov 24 03:55:07 bhuna 00000000
Nov 24 03:55:07 bhuna 00004000
Nov 24 03:55:07 bhuna 00000000
Nov 24 03:55:07 bhuna
Nov 24 03:55:07 bhuna [233316.039475]
Nov 24 03:55:08 bhuna 00000000
Nov 24 03:55:08 bhuna 00000000
Nov 24 03:55:08 bhuna 00000000
Nov 24 03:55:08 bhuna 0000000f
Nov 24 03:55:08 bhuna e27dad70
Nov 24 03:55:08 bhuna 00000000
Nov 24 03:55:08 bhuna 00000086
Nov 24 03:55:08 bhuna 00000246
Nov 24 03:55:08 bhuna
Nov 24 03:55:08 bhuna [233316.039475] Call Trace:
Nov 24 03:55:08 bhuna [233316.039475] [<c016a0dc>]
Nov 24 03:55:08 bhuna bounce_end_io_read+0x0/0xd
Nov 24 03:55:08 bhuna [233316.039475] [<c0192d38>]
Nov 24 03:55:08 bhuna bio_endio+0x26/0x28
Nov 24 03:55:08 bhuna [233316.039475] [<c01d1de3>]
Nov 24 03:55:08 bhuna __end_that_request_first+0x1b2/0x2c3
Nov 24 03:55:08 bhuna [233316.039475] [<c01d2712>]
Nov 24 03:55:08 bhuna __blk_end_request+0x16/0x37
Nov 24 03:55:08 bhuna [233316.039475] [<f880a293>]
Nov 24 03:55:08 bhuna blk_done+0x3d/0x88 [virtio_blk]
Nov 24 03:55:08 bhuna [233316.039475] [<c024c6cb>]
Nov 24 03:55:08 bhuna vring_interrupt+0x23/0x2e
Nov 24 03:55:08 bhuna [233316.039475] [<f882f0a6>]
Nov 24 03:55:08 bhuna vp_interrupt+0x67/0xa4 [virtio_pci]
Nov 24 03:55:08 bhuna [233316.039475] [<c01520ea>]
Nov 24 03:55:08 bhuna handle_IRQ_event+0x23/0x51
Nov 24 03:55:08 bhuna [233316.039475] [<c01531e9>]
Nov 24 03:55:08 bhuna handle_fasteoi_irq+0x71/0xa4
Nov 24 03:55:08 bhuna [233316.039475] [<c0105f3a>]
Nov 24 03:55:08 bhuna do_IRQ+0x4d/0x63
Nov 24 03:55:08 bhuna [233316.039475] [<c01042a7>]
Nov 24 03:55:08 bhuna common_interrupt+0x23/0x28
Nov 24 03:55:08 bhuna [233316.039475] [<c01265e5>]
Nov 24 03:55:08 bhuna __do_softirq+0x56/0xd3
Nov 24 03:55:08 bhuna [233316.039475] [<c01266a7>]
Nov 24 03:55:08 bhuna do_softirq+0x45/0x53
Nov 24 03:55:08 bhuna [233316.039475] [<c012695e>]
Nov 24 03:55:08 bhuna irq_exit+0x35/0x67
Nov 24 03:55:08 bhuna [233316.039475] [<c01101c9>]
Nov 24 03:55:08 bhuna smp_apic_timer_interrupt+0x6b/0x76
Nov 24 03:55:08 bhuna [233316.039475] [<c0104364>]
Nov 24 03:55:08 bhuna apic_timer_interrupt+0x28/0x30
Nov 24 03:55:08 bhuna [233316.039475] [<c0115009>]
Nov 24 03:55:08 bhuna paravirt_leave_lazy+0x1c/0x1d
Nov 24 03:55:08 bhuna [233316.039475] [<c0114b44>]
Nov 24 03:55:08 bhuna kvm_leave_lazy_mmu+0x24/0x31
Nov 24 03:55:08 bhuna [233316.039475] [<c0162a80>]
Nov 24 03:55:08 bhuna unmap_vmas+0x3f0/0x5ad
Nov 24 03:55:08 bhuna [233316.039475] [<c01dffd0>]
Nov 24 03:55:08 bhuna rb_insert_color+0x4c/0xad
Nov 24 03:55:08 bhuna [233316.039475] [<c0165fbb>]
Nov 24 03:55:08 bhuna vma_adjust+0x310/0x399
Nov 24 03:55:08 bhuna [233316.039475] [<c016587d>]
Nov 24 03:55:08 bhuna unmap_region+0x83/0xe9
Nov 24 03:55:08 bhuna [233316.039475] [<c016625f>]
Nov 24 03:55:08 bhuna do_munmap+0x148/0x19b
Nov 24 03:55:08 bhuna [233316.039475] [<c01668cf>]
Nov 24 03:55:08 bhuna mmap_region+0x6f/0x393
Nov 24 03:55:08 bhuna [233316.039475] [<c0166f87>]
Nov 24 03:55:09 bhuna arch_get_unmapped_area_topdown+0x0/0x120
Nov 24 03:55:09 bhuna [233316.039475] [<c0166f34>]
Nov 24 03:55:09 bhuna do_mmap_pgoff+0x266/0x2b9
Nov 24 03:55:09 bhuna [233316.039475] [<c0106a4e>]
Nov 24 03:55:09 bhuna sys_mmap2+0x86/0xa0
Nov 24 03:55:09 bhuna [233316.039475] [<c01038ce>]
Nov 24 03:55:09 bhuna syscall_call+0x7/0xb
Nov 24 03:55:09 bhuna [233316.039475] =======================
Nov 24 03:55:09 bhuna [233316.039475] Code:
Nov 24 03:55:09 bhuna 8d
Nov 24 03:55:09 bhuna 04
Nov 24 03:55:09 bhuna 05
Nov 24 03:55:09 bhuna 00
Nov 24 03:55:09 bhuna 00
Nov 24 03:55:09 bhuna 00
Nov 24 03:55:09 bhuna 00
Nov 24 03:55:09 bhuna 90
Nov 24 03:55:09 bhuna 89
Nov 24 03:55:09 bhuna c3
Nov 24 03:55:09 bhuna fa
Nov 24 03:55:09 bhuna 8d
Nov 24 03:55:09 bhuna 04
Nov 24 03:55:09 bhuna 05
Nov 24 03:55:09 bhuna 00
Nov 24 03:55:09 bhuna 00
Nov 24 03:55:09 bhuna 00
Nov 24 03:55:09 bhuna 00
Nov 24 03:55:09 bhuna 90
Nov 24 03:55:09 bhuna 90
Nov 24 03:55:09 bhuna 8b
Nov 24 03:55:09 bhuna 45
Nov 24 03:55:09 bhuna f4
Nov 24 03:55:09 bhuna 31
Nov 24 03:55:09 bhuna d2
Nov 24 03:55:09 bhuna e8
Nov 24 03:55:09 bhuna 80
Nov 24 03:55:09 bhuna db
Nov 24 03:55:09 bhuna fa
Nov 24 03:55:09 bhuna ff
Nov 24 03:55:09 bhuna 8b
Nov 24 03:55:09 bhuna 55
Nov 24 03:55:09 bhuna f8
Nov 24 03:55:09 bhuna 89
Nov 24 03:55:09 bhuna d1
Nov 24 03:55:09 bhuna c1
Nov 24 03:55:09 bhuna e9
Nov 24 03:55:09 bhuna 02
Nov 24 03:55:09 bhuna 89
Nov 24 03:55:09 bhuna c7
Nov 24 03:55:09 bhuna 03
Nov 24 03:55:09 bhuna 7d
Nov 24 03:55:09 bhuna fc
Nov 24 03:55:09 bhuna a5
Nov 24 03:55:09 bhuna 89
Nov 24 03:55:09 bhuna d1
Nov 24 03:55:09 bhuna 83
Nov 24 03:55:09 bhuna e1
Nov 24 03:55:09 bhuna 03
Nov 24 03:55:09 bhuna 74
Nov 24 03:55:09 bhuna 02
Nov 24 03:55:09 bhuna f3
Nov 24 03:55:09 bhuna a4
Nov 24 03:55:09 bhuna 31
Nov 24 03:55:09 bhuna d2
Nov 24 03:55:09 bhuna e8
Nov 24 03:55:09 bhuna f4
Nov 24 03:55:09 bhuna db
Nov 24 03:55:09 bhuna fa
Nov 24 03:55:09 bhuna ff
Nov 24 03:55:09 bhuna 89
Nov 24 03:55:09 bhuna d8
Nov 24 03:55:09 bhuna 50
Nov 24 03:55:09 bhuna
Nov 24 03:55:09 bhuna [233316.039475] EIP: [<c016a080>]
Nov 24 03:55:09 bhuna __bounce_end_io_read+0x88/0xd7
Nov 24 03:55:09 SS: ESP 0068:c1c91cd4
Nov 24 03:55:09 bhuna [233316.039475] Kernel panic - not syncing: Fatal exception in interrupt
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
Reply to: