Bug#648811: NULL pointer dereference in Linux 2.6.32-5-686 : ocfs2_update_disk_slot when peer hang
Package: linux-2.6
Version: 2.6.32-38
Severity: important
Tags: squeeze
Hello there!
Cluster setup:
Two virtual machine (qemu and/or vmware esx)
Debian squeeze 6.0.3 with stock kernel
o2cb_ctl version 1.4.4
Shared volume setup on top of drbd (dual primary)
Howto reproduce:
File system created and mounted on one peer.
When it dies, the other peer mount the volume and it hang with following trace:
Note:
Bug already reported on oracle oss bugtracker (http://oss.oracle.com/bugzilla/show_bug.cgi?id=1335)
-- Package-specific info:
** Version:
Linux version 2.6.32-5-686 (Debian 2.6.32-38) (ben@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 SMP Mon Oct 3 04:15:24 UTC 2011
** Command line:
BOOT_IMAGE=/vmlinuz-2.6.32-5-686 root=/dev/mapper/vg00-lvroot ro quiet vga=770
** Not tainted
** Kernel log:
[ 1590.801556] BUG: unable to handle kernel NULL pointer dereference at 0000001e
[ 1590.803047] IP: [<e163a869>] ocfs2_update_disk_slot+0x6b/0x121 [ocfs2]
[ 1590.804242] *pde = 00000000
[ 1590.805399] Oops: 0000 [#1] SMP
[ 1590.805517] last sysfs file: /sys/fs/o2cb/interface_revision
[ 1590.805517] Modules linked in: ocfs2 jbd2 quota_tree sha1_generic ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs drbd lru_cache cn fuse ext2 loop
snd_pcm snd_timer parport_pc snd i2c_piix4 parport soundcore snd_page_alloc psmouse processor i2c_core button pcspkr evdev serio_raw ext3 jbd mbcache dm_mod sg sd_mod crc_t10dif sr_mod cdrom ata_generic ata_piix libata
thermal thermal_sys floppy 8139too 8139cp mii scsi_mod [last unloaded: configfs]
[ 1590.805517]
[ 1590.805517] Pid: 4265, comm: ocfs2rec Not tainted (2.6.32-5-686 #1) Bochs
[ 1590.805517] EIP: 0060:[<e163a869>] EFLAGS: 00010246 CPU: 0
[ 1590.805517] EIP is at ocfs2_update_disk_slot+0x6b/0x121 [ocfs2]
[ 1590.805517] EAX: 00000006 EBX: d946df40 ECX: 00000000 EDX: 00000000
[ 1590.805517] ESI: 00000000 EDI: d7d73800 EBP: dde30800 ESP: d6999ebc
[ 1590.805517] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1590.805517] Process ocfs2rec (pid: 4265, ti=d6998000 task=ddd8aec0 task.ti=d6998000)
[ 1590.805517] Stack:
[ 1590.805517] e163abff dde30800 00000000 d7d73800 00000000 e16210a0 ddd8aec0 ddd8aec0
[ 1590.805517] <0> dde3089c 00000006 00000000 dde30800 da26a108 da26a118 00000001 00010000
[ 1590.805517] <0> 00000000 00010000 00000000 dad1a000 df707d48 00000000 00000000 00000000
[ 1590.805517] Call Trace:
[ 1590.805517] [<e163abff>] ? ocfs2_clear_slot+0x23/0x50 [ocfs2]
[ 1590.805517] [<e16210a0>] ? __ocfs2_recovery_thread+0xfd5/0x146d [ocfs2]
[ 1590.805517] [<e16200cb>] ? __ocfs2_recovery_thread+0x0/0x146d [ocfs2]
[ 1590.805517] [<c1043dfc>] ? kthread+0x61/0x66
[ 1590.805517] [<c1043d9b>] ? kthread+0x0/0x66
[ 1590.805517] [<c1003d47>] ? kernel_thread_helper+0x7/0x10
[ 1590.805517] Code: 18 8b 43 18 8b 04 f0 8b 34 24 88 04 f7 03 53 18 83 3a 00 74 07 8b 42 04 89 44 f7 04 8b 43 10 8b 14 88 eb 35 8b 43 10 31 d2 8b 00 <8b> 48 18 eb 1f 8d 04 d5 00 00 00
00 03 43 18 83 38 00 74 09 8b
[ 1590.805517] EIP: [<e163a869>] ocfs2_update_disk_slot+0x6b/0x121 [ocfs2] SS:ESP 0068:d6999ebc
[ 1590.805517] CR2: 000000000000001e
[ 1590.846312] ---[ end trace eee3645cbe324c05 ]---
** Model information
not available
** Loaded modules:
Module Size Used by
drbd 173348 0
lru_cache 4054 1 drbd
cn 3667 1 drbd
fuse 44268 1
ocfs2_dlmfs 12533 1
ocfs2_stack_o2cb 2526 0
ocfs2_dlm 157108 2 ocfs2_dlmfs,ocfs2_stack_o2cb
ocfs2_nodemanager 125027 3 ocfs2_dlmfs,ocfs2_stack_o2cb,ocfs2_dlm
ocfs2_stackglue 5625 1 ocfs2_stack_o2cb
configfs 15888 2 ocfs2_nodemanager
ext2 46245 1
loop 9769 0
snd_pcm 47226 0
snd_timer 12270 1 snd_pcm
snd 34423 2 snd_pcm,snd_timer
soundcore 3450 1 snd
snd_page_alloc 5045 1 snd_pcm
parport_pc 15799 0
pcspkr 1207 0
parport 22554 1 parport_pc
i2c_piix4 7076 0
i2c_core 12787 1 i2c_piix4
processor 26327 0
button 3598 0
psmouse 44809 0
evdev 5609 2
serio_raw 2916 0
ext3 94396 3
jbd 32317 1 ext3
mbcache 3762 2 ext2,ext3
dm_mod 46234 14
sg 19937 0
sd_mod 26005 3
crc_t10dif 1012 1 sd_mod
sr_mod 10770 0
cdrom 26487 1 sr_mod
ata_generic 2247 0
ata_piix 17736 2
libata 115869 2 ata_generic,ata_piix
thermal 9206 0
floppy 40923 0
thermal_sys 9378 2 processor,thermal
8139too 14949 0
8139cp 13421 0
mii 2714 2 8139too,8139cp
scsi_mod 104853 4 sg,sd_mod,sr_mod,libata
** PCI devices:
00:00.0 Host bridge [0600]: Intel Corporation 440FX - 82441FX PMC [Natoma] [8086:1237] (rev 02)
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
00:01.0 ISA bridge [0601]: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] [8086:7000]
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
00:01.1 IDE interface [0101]: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] [8086:7010] (prog-if 80 [Master])
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Region 0: [virtual] Memory at 000001f0 (32-bit, non-prefetchable) [size=8]
Region 1: [virtual] Memory at 000003f0 (type 3, non-prefetchable) [size=1]
Region 2: [virtual] Memory at 00000170 (32-bit, non-prefetchable) [size=8]
Region 3: [virtual] Memory at 00000370 (type 3, non-prefetchable) [size=1]
Region 4: I/O ports at c000 [size=16]
Kernel driver in use: ata_piix
00:01.3 Bridge [0680]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI [8086:7113] (rev 03)
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 9
Kernel driver in use: piix4_smbus
00:02.0 VGA compatible controller [0300]: Cirrus Logic GD 5446 [1013:00b8] (prog-if 00 [VGA controller])
Subsystem: Red Hat, Inc Device [1af4:1100]
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Region 0: Memory at f0000000 (32-bit, prefetchable) [size=32M]
Region 1: Memory at f2000000 (32-bit, non-prefetchable) [size=4K]
Expansion ROM at f2010000 [disabled] [size=64K]
00:03.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ [10ec:8139] (rev 20)
Subsystem: Red Hat, Inc Device [1af4:1100]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 11
Region 0: I/O ports at c100 [size=256]
Region 1: Memory at f2020000 (32-bit, non-prefetchable) [size=256]
Expansion ROM at f2030000 [disabled] [size=64K]
Kernel driver in use: 8139cp
** USB devices:
not available
-- System Information:
Debian Release: 6.0.3
APT prefers stable
APT policy: (500, 'stable')
Architecture: i386 (i686)
Kernel: Linux 2.6.32-5-686 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages linux-image-2.6.32-5-686 depends on:
ii debconf [debconf-2.0] 1.5.36.1 Debian configuration management sy
ii initramfs-tools [linux-initra 0.98.8 tools for generating an initramfs
ii linux-base 2.6.32-38 Linux image base package
ii module-init-tools 3.12-1 tools for managing Linux kernel mo
Versions of packages linux-image-2.6.32-5-686 recommends:
pn firmware-linux-free <none> (no description available)
ii libc6-i686 2.11.2-10 Embedded GNU C Library: Shared lib
Versions of packages linux-image-2.6.32-5-686 suggests:
pn grub | lilo <none> (no description available)
pn linux-doc-2.6.32 <none> (no description available)
Versions of packages linux-image-2.6.32-5-686 is related to:
pn firmware-bnx2 <none> (no description available)
pn firmware-bnx2x <none> (no description available)
pn firmware-ipw2x00 <none> (no description available)
pn firmware-ivtv <none> (no description available)
pn firmware-iwlwifi <none> (no description available)
pn firmware-linux <none> (no description available)
pn firmware-linux-nonfree <none> (no description available)
pn firmware-qlogic <none> (no description available)
pn firmware-ralink <none> (no description available)
pn xen-hypervisor <none> (no description available)
-- debconf information:
linux-image-2.6.32-5-686/prerm/removing-running-kernel-2.6.32-5-686: true
linux-image-2.6.32-5-686/postinst/depmod-error-initrd-2.6.32-5-686: false
linux-image-2.6.32-5-686/postinst/ignoring-do-bootloader-2.6.32-5-686:
linux-image-2.6.32-5-686/postinst/missing-firmware-2.6.32-5-686:
Reply to: