rcu_sched self-detected stall on CPU { 0}
Hello,
I do not remember when I see last term syslogd printing messages on the
terminal in X (konsole in KDE to be precise). But I do remember that these
messages indicated serious issues :0(
The worst thing is I do not even understand what the system wanted to tell me.
>From the syslog, I can speculate on a rcu_sched self-detect stall (therefore I
copied debian-kernel). I am running kernel 3.13 since a few days. Can anybody
help here?
I attach the console output and the relevant section of /var/log/syslog
Console output:
Message from syslogd@www at Fri Mar 21 14:28:50 2014 ...
www vmunix: [118396.260001] CPU: 0 PID: 29 Comm: kswapd0 Tainted: G C
O 3.13-1-686-pae #1 Debian 3.13.5-1
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260001] Hardware name: System manufacturer P5E-VM
HDMI/P5E-VM HDMI, BIOS 0405 12/17/2007
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260001] task: f75344a0 ti: f76d6000 task.ti: f76d6000
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260001] Stack:
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260001] Call Trace:
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260001] Code: 00 48 75 fd 48 c3 8d 74 26 00 8d bc 27 00 00
00 00 ff 15 30 0c 59 c1 f3 c3 90 8d b4 26 00 00 00 00 c1 e0 02 64 8b 15 9c be
63 c1 <6b> d2 3e f7 e2 8d 42 01 ff 15 30 0c 59 c1 f3 c3 8d b6 00 00 00
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260211] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G C
O 3.13-1-686-pae #1 Debian 3.13.5-1
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260214] Hardware name: System manufacturer P5E-VM
HDMI/P5E-VM HDMI, BIOS 0405 12/17/2007
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260216] task: f7494da0 ti: f749c000 task.ti: f749c000
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260233] Stack:
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260250] Call Trace:
Message from syslogd@www at Fri Mar 21 14:29:10 2014 ...
www vmunix: [118396.260268] Code: 00 00 8d bc 27 00 00 00 00 fa c3 8d b4 26 00
00 00 00 8d bc 27 00 00 00 00 fb c3 8d b4 26 00 00 00 00 8d bc 27 00 00 00 00
fb f4 <c3> 8d b6 00 00 00 00 8d bc 27 00 00 00 00 f4 c3 8d b4 26 00 00
Content of syslog:
Mar 21 14:23:53 www ddclient[3535]: WARNING: file /etc/ddclient.conf, line 8:
Invalid Value for keyword 'login' = ''
Mar 21 14:26:50 www vmunix: [118276.128002] [sched_delayed] sched: RT
throttling activated
Mar 21 14:28:50 www vmunix: [118396.260001] INFO: rcu_sched self-detected
stall on CPU { 0} (t=5250 jiffies g=2105139 c=2105138 q=2179)
Mar 21 14:28:50 www vmunix: [118396.260001] sending NMI to all CPUs:
Mar 21 14:28:50 www vmunix: [118396.260001] NMI backtrace for cpu 0
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260001] CPU: 0 PID: 29 Comm: kswapd0
Tainted: G C O 3.13-1-686-pae #1 Debian 3.13.5-1
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260001] Hardware name: System manufacturer
P5E-VM HDMI/P5E-VM HDMI, BIOS 0405 12/17/2007
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260001] task: f75344a0 ti: f76d6000
task.ti: f76d6000
Mar 21 14:29:10 www vmunix: [118396.260001] EIP: 0060:[<c122173a>] EFLAGS:
00000006 CPU: 0
Mar 21 14:29:10 www vmunix: [118396.260001] EIP is at __const_udelay+0xa/0x20
Mar 21 14:29:10 www vmunix: [118396.260001] EAX: 01062560 EBX: 00002710 ECX:
fffff000 EDX: 00a30dac
Mar 21 14:29:10 www vmunix: [118396.260001] ESI: c1576440 EDI: c1576440 EBP:
00000000 ESP: f76d7b7c
Mar 21 14:29:10 www vmunix: [118396.260001] DS: 007b ES: 007b FS: 00d8 GS:
00e0 SS: 0068
Mar 21 14:29:10 www vmunix: [118396.260001] CR0: 8005003b CR2: b6930d40 CR3:
2c03a000 CR4: 000007f0
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260001] Stack:
Mar 21 14:29:10 www vmunix: [118396.260001] c10392ed c14cccc9 f79b7840
c109a39a c14dad94 00001482 00201f33 00201f32
Mar 21 14:29:10 www vmunix: [118396.260001] 00000883 00006bae f79b7840
c1576440 c15b31ac 00000000 00000000 00000082
Mar 21 14:29:10 www vmunix: [118396.260001] f75344a0 00000000 00000000
00006bae c1059044 f76d7ca0 f76d7ca0 47052501
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260001] Call Trace:
Mar 21 14:29:10 www vmunix: [118396.260001] [<c10392ed>] ?
arch_trigger_all_cpu_backtrace+0x4d/0x70
Mar 21 14:29:10 www vmunix: [118396.260001] [<c109a39a>] ?
rcu_check_callbacks+0x37a/0x5b0
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1059044>] ?
update_process_times+0x34/0x60
Mar 21 14:29:10 www vmunix: [118396.260001] [<c10a48dc>] ?
tick_sched_handle.isra.12+0x1c/0x50
Mar 21 14:29:10 www vmunix: [118396.260001] [<c10a493f>] ?
tick_sched_timer+0x2f/0x60
Mar 21 14:29:10 www vmunix: [118396.260001] [<c106b487>] ?
__remove_hrtimer+0x27/0x80
Mar 21 14:29:10 www vmunix: [118396.260001] [<c106b68b>] ?
__run_hrtimer+0x6b/0x190
Mar 21 14:29:10 www vmunix: [118396.260001] [<c10a4910>] ?
tick_sched_handle.isra.12+0x50/0x50
Mar 21 14:29:10 www vmunix: [118396.260001] [<c106c2a1>] ?
hrtimer_interrupt+0x201/0x2c0
Mar 21 14:29:10 www vmunix: [118396.260001] [<c11f6b03>] ?
blk_queue_bio+0x103/0x2e0
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1037467>] ?
local_apic_timer_interrupt+0x27/0x50
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1051e5d>] ? irq_enter+0xd/0x60
Mar 21 14:29:10 www vmunix: [118396.260001] [<c10375fb>] ?
smp_apic_timer_interrupt+0x2b/0x50
Mar 21 14:29:10 www vmunix: [118396.260001] [<c140fbec>] ?
apic_timer_interrupt+0x34/0x3c
Mar 21 14:29:10 www vmunix: [118396.260001] [<c104007b>] ?
kvm_read_and_reset_pf_reason+0x2b/0x40
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1127420>] ?
scan_swap_map+0x120/0x400
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1127819>] ?
get_swap_page+0x119/0x150
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1124c9c>] ?
add_to_swap+0xc/0x60
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1102351>] ?
shrink_page_list+0x4e1/0x8d0
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1102c7a>] ?
shrink_inactive_list+0x16a/0x460
Mar 21 14:29:10 www vmunix: [118396.260001] [<c110353a>] ?
shrink_lruvec+0x2ca/0x5e0
Mar 21 14:29:10 www vmunix: [118396.260001] [<c10736ed>] ?
ttwu_do_wakeup+0xd/0xf0
Mar 21 14:29:10 www vmunix: [118396.260001] [<c11038a0>] ?
shrink_zone+0x50/0x160
Mar 21 14:29:10 www vmunix: [118396.260001] [<c11046f7>] ? kswapd+0x487/0x950
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1104270>] ?
mem_cgroup_shrink_node_zone+0x100/0x100
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1068c6e>] ? kthread+0x9e/0xc0
Mar 21 14:29:10 www vmunix: [118396.260001] [<c14161f7>] ?
ret_from_kernel_thread+0x1b/0x28
Mar 21 14:29:10 www vmunix: [118396.260001] [<c1068bd0>] ?
kthread_create_on_node+0x110/0x110
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260001] Code: 00 48 75 fd 48 c3 8d 74 26
00 8d bc 27 00 00 00 00 ff 15 30 0c 59 c1 f3 c3 90 8d b4 26 00 00 00 00 c1 e0
02 64 8b 15 9c be 63 c1 <6b> d2 3e f7 e2 8d 42 01 ff 15 30 0c 59 c1 f3 c3 8d b6
00 00 00
Mar 21 14:29:10 www vmunix: [118396.260207] NMI backtrace for cpu 1
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260211] CPU: 1 PID: 0 Comm: swapper/1
Tainted: G C O 3.13-1-686-pae #1 Debian 3.13.5-1
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260214] Hardware name: System manufacturer
P5E-VM HDMI/P5E-VM HDMI, BIOS 0405 12/17/2007
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260216] task: f7494da0 ti: f749c000
task.ti: f749c000
Mar 21 14:29:10 www vmunix: [118396.260219] EIP: 0060:[<c1040692>] EFLAGS:
00200286 CPU: 1
Mar 21 14:29:10 www vmunix: [118396.260223] EIP is at
native_safe_halt+0x2/0x10
Mar 21 14:29:10 www vmunix: [118396.260225] EAX: ffffffed EBX: edb4421f ECX:
01000000 EDX: 00000000
Mar 21 14:29:10 www vmunix: [118396.260227] ESI: f749c000 EDI: 00000001 EBP:
00000000 ESP: f749df68
Mar 21 14:29:10 www vmunix: [118396.260229] DS: 007b ES: 007b FS: 00d8 GS:
00e0 SS: 0068
Mar 21 14:29:10 www vmunix: [118396.260231] CR0: 8005003b CR2: 089510e0 CR3:
2ce94000 CR4: 000007f0
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260233] Stack:
Mar 21 14:29:10 www vmunix: [118396.260234] c1015cd4 edb4421f f749c000
00000000 c1016396 c10919a9 10fe296c edb4421f
Mar 21 14:29:10 www vmunix: [118396.260240] 4d11762a 00000000 00000000
00000000 c1035b8e ffffffff 00000000 00000000
Mar 21 14:29:10 www vmunix: [118396.260245] 00000000 9da374a5 4d11762a
01020800 00000000 00000000 00000000 00000000
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260250] Call Trace:
Mar 21 14:29:10 www vmunix: [118396.260255] [<c1015cd4>] ?
default_idle+0x14/0xa0
Mar 21 14:29:10 www vmunix: [118396.260258] [<c1016396>] ?
arch_cpu_idle+0x16/0x20
Mar 21 14:29:10 www vmunix: [118396.260263] [<c10919a9>] ?
cpu_startup_entry+0x1b9/0x200
Mar 21 14:29:10 www vmunix: [118396.260266] [<c1035b8e>] ?
start_secondary+0x1ce/0x240
Mar 21 14:29:10 www syslogd: /dev/:0: No such file or directory
Mar 21 14:29:10 www syslogd: /dev/:1: No such file or directory
Mar 21 14:29:10 www vmunix: [118396.260268] Code: 00 00 8d bc 27 00 00 00 00
fa c3 8d b4 26 00 00 00 00 8d bc 27 00 00 00 00 fb c3 8d b4 26 00 00 00 00 8d
bc 27 00 00 00 00 fb f4 <c3> 8d b6 00 00 00 00 8d bc 27 00 00 00 00 f4 c3 8d
b4 26 00 00
Mar 21 14:29:10 www ddclient[3535]: WARNING: file /etc/ddclient.conf, line 8:
Invalid Value for keyword 'login' = ''
Mar 21 14:30:25 www vmunix: [118490.575735] [drm:intel_crtc_cursor_set],
cursor off
Any hint is welcome.
Many thanks,
Rainer
--
Rainer Dorsch
http://bokomoko.de/
Reply to: