Bug#794060: linux-image-3.2.0-4-686-pae: kernel oops in do_ip_vs_get_ctl
Package: linux-image-3.2.0-4-686-pae
Version: 3.2.68-1+deb7u2
Severity: normal
Dear Maintainer,
Using keepalived for IPVS, sometimes during bootup an kernel oops gets triggered.
It seems to be a race condition when keepalived is accessing IPVS while later isn't fully initialized yet.
Jul 30 07:11:16 rtr kernel: : [ 276.388953] Pid: 14847, comm: keepalived Tainted: G C 3.2.0-4-686-pae #1 Debian 3.2.68-1+deb7u2 /i852GM-W83627THF
Jul 30 07:11:16 rtr kernel: : [ 276.388953] EIP: 0060:[<e099b1a0>] EFLAGS: 00210293 CPU: 0
Jul 30 07:11:16 rtr kernel: : [ 276.388953] EIP is at do_ip_vs_get_ctl+0x206/0x54d [ip_vs]
Jul 30 07:11:16 rtr kernel: : [ 276.388953] EAX: 00001000 EBX: 00000000 ECX: 0000000c EDX: df46dd8c
Jul 30 07:11:16 rtr kernel: : [ 276.388953] ESI: df46df30 EDI: fffffe00 EBP: e09a9480 ESP: df46dd7c
Jul 30 07:11:16 rtr kernel: : [ 276.388953] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Jul 30 07:11:16 rtr kernel: : [ 276.388953] Process keepalived (pid: 14847, ti=df46c000 task=df58a570 task.ti=df46c000)
Jul 30 07:11:16 rtr kernel: : [ 276.388953] Stack:
Jul 30 07:11:16 rtr kernel: : [ 276.388953] c140dd80 08073c34 00000481 00000001 00010201 00001000 df46df5c 00000000
Jul 30 07:11:16 rtr kernel: : [ 276.388953] 00000000 00000000 c12c5920 de94e5b4 c12169ea de94e580 ffffffa4 00200246
Jul 30 07:11:16 rtr kernel: : [ 276.388953] c103d213 c12479bf 08073c34 bf8c20ac 00000002 00000000 df46de10 00415500
Jul 30 07:11:16 rtr kernel: : [ 276.388953] Call Trace:
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c12c5920>] ? _raw_spin_lock_bh+0x9/0x12
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c12169ea>] ? release_sock+0x10/0xc2
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c103d213>] ? _local_bh_enable_ip.isra.9+0x15/0x6d
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c12479bf>] ? do_ip_getsockopt.constprop.10+0x3db/0x48a
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c116744c>] ? _copy_from_user+0x28/0x47
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c12c483a>] ? _cond_resched+0x5/0x18
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c116744c>] ? _copy_from_user+0x28/0x47
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c115f230>] ? cpumask_any_but+0x1e/0x2a
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c1028728>] ? flush_tlb_page+0x46/0x5d
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c1027a6e>] ? ptep_set_access_flags+0x4c/0x54
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c10ad349>] ? do_wp_page+0x2f3/0x613
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c10aeeed>] ? handle_pte_fault+0x863/0x8dc
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c123e8de>] ? nf_sockopt+0xef/0x119
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c123e91a>] ? nf_getsockopt+0x12/0x16
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c1247ad3>] ? ip_getsockopt+0x65/0x91
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c1216352>] ? sock_common_getsockopt+0x12/0x16
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c1215b8b>] ? sys_getsockopt+0x65/0x82
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c121610c>] ? sys_socketcall+0x176/0x1cd
Jul 30 07:11:16 rtr kernel: : [ 276.388953] [<c12c59ec>] ? syscall_call+0x7/0x7
Jul 30 07:11:16 rtr kernel: : [ 276.388953] Code: 00 00 89 d8 e8 a8 bc 7c e0 40 89 06 e9 2c 03 00 00 a1 f8 88 9a e0 b9 0c 00 00 00 c7 44 24 10 01 02 01 00 8d 54 24 10 89 44 24 14 <8b> 83 44 03 00 00 89 44 24 18 e9 e7 02 00 00 6b 54 24 64 60 8b
Jul 30 07:11:16 rtr kernel: : [ 276.388953] EIP: [<e099b1a0>] do_ip_vs_get_ctl+0x206/0x54d [ip_vs] SS:ESP 0068:df46dd7c
Jul 30 07:11:16 rtr kernel: : [ 276.388953] Code: 00 00 89 d8 e8 a8 bc 7c e0 40 89 06 e9 2c 03 00 00 a1 f8 88 9a e0 b9 0c 00 00 00 c7 44 24 10 01 02 01 00 8d 54 24 10 89 44 24 14 <8b> 83 44 03 00 00 89 44 24 18 e9 e7 02 00 00 6b 54 24 64 60 8b
Jul 30 07:11:16 rtr kernel: : [ 276.388953] EIP: [<e099b1a0>] do_ip_vs_get_ctl+0x206/0x54d [ip_vs] SS:ESP 0068:df46dd7c
Jul 30 07:11:16 rtr kernel: : [ 276.388953] CR2: 0000000000000344
Jul 30 07:11:16 rtr kernel: : [ 276.926199] ---[ end trace 7d43c0c3956303e6 ]---
Jul 30 07:11:16 rtr kernel: : [ 276.938589] IPVS: Connection hash table configured (size=4096, memory=32Kbytes)
Jul 30 07:11:16 rtr kernel: : [ 276.972690] IPVS: Creating netns size=1220 id=0
Jul 30 07:11:16 rtr kernel: : [ 276.986029] IPVS: ipvs loaded.
keepalived processes are then stuck in device state and only a reboot helps to recover.
Problem is known here [1] and for Redhat [2].
Patch is available upstream [3] and seems to be included from kernel 3.4 on.
Any chance to get this fix into the Wheezy kernel?
Cheers,
Andreas
[1] http://www.spinics.net/lists/lvs-devel/msg02349.html
[2] https://bugzilla.redhat.com/show_bug.cgi?id=806704
[3] http://archive.linuxvirtualserver.org/html/lvs-devel/2012-04/msg00120.html
Reply to: