Bug#602273: linux-image-2.6.32-5-686-bigmem: domU hangs during dom0 reboot, recovers when dom0 uptime "caught up"
Package: linux-image-2.6.32-5-686-bigmem
Version: 2.6.32-26
Severity: normal
Steps to reproduce:
1) use 64-bit squeeze as dom0
2) use 32-bit squeeze as domU
3) reboot dom0
Expected results:
3) domU's state is saved to disk and then restored
Actual results:
2) domU's state is saved to disk but it hangs after restore
More info:
1) Saving and restoring the domU with "xm save" and "xm restore"
worked at least twice. The problem occurs only if dom0 is rebooted.
=> Is some clock perhaps visible to the domU and which then gets
confused when it is reset and thus goes backwards in time?
2) hypervisor and dom0 information:
overlord2:~$ dpkg-query -W xen-hypervisor-4.0-amd64 linux-image-$(uname -r)
linux-image-2.6.32-5-xen-amd64 2.6.32-27
xen-hypervisor-4.0-amd64 4.0.1-1
overlord2:~$ sudo xm info
host : overlord2
release : 2.6.32-5-xen-amd64
version : #1 SMP Wed Oct 20 02:22:18 UTC 2010
machine : x86_64
nr_cpus : 4
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2394
hw_caps : bfebfbff:20100800:00000000:00000940:0000e3bd:00000000:00000001:00000000
virt_caps : hvm hvm_directio
total_memory : 8189
free_memory : 3
node_to_cpu : node0:0-3
node_to_memory : node0:3
node_to_dma32_mem : node0:3
max_node_id : 0
xen_major : 4
xen_minor : 0
xen_extra : .1
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline :
cc_compiler : gcc version 4.4.5 20100824 (prerelease) (Debian 4.4.4-11)
cc_compile_by : waldi
cc_compile_domain : debian.org
cc_compile_date : Fri Sep 3 15:38:12 UTC 2010
xend_config_format : 4
overlord2:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
stepping : 11
cpu MHz : 2394.082
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good aperfmperf pni est ssse3 cx16 hypervisor lahf_lm
bogomips : 4788.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
stepping : 11
cpu MHz : 2394.082
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good aperfmperf pni est ssse3 cx16 hypervisor lahf_lm
bogomips : 4788.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
stepping : 11
cpu MHz : 2394.082
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good aperfmperf pni est ssse3 cx16 hypervisor lahf_lm
bogomips : 4788.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
stepping : 11
cpu MHz : 2394.082
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good aperfmperf pni est ssse3 cx16 hypervisor lahf_lm
bogomips : 4788.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
3) domU information:
$ dpkg-query -W linux-image-$(uname -r)
linux-image-2.6.32-5-686-bigmem 2.6.32-26
4) We used
sudo xm dump-core squeeze32 squeeze32.dump
to save the memory of the stuck domU to disk.
5) We used
crash -s --no_panic -e emacs vmlinux-2.6.32-5-686-bigmem squeeze32.dump
to analyze the memory dump:
crash> sys
KERNEL: vmlinux-2.6.32-5-686-bigmem
DUMPFILE: D32.2
CPUS: 4
DATE: Tue Nov 2 17:44:23 2010
UPTIME: 00:11:05
LOAD AVERAGE: 0.00, 0.00, 0.00
TASKS: 79
NODENAME: squeeze32
RELEASE: 2.6.32-5-686-bigmem
VERSION: #1 SMP Tue Oct 19 15:03:03 UTC 2010
MACHINE: i686 (2394 Mhz)
MEMORY: 511.6 MB
PANIC: ""
crash> ps | grep -v IN
PID PPID CPU TASK ST %MEM VSZ RSS COMM
> 0 0 0 c13a2ba0 RU 0.0 0 0 [swapper]
> 0 0 1 df830880 RU 0.0 0 0 [swapper]
> 0 0 2 df830cc0 RU 0.0 0 0 [swapper]
> 0 0 3 df831100 RU 0.0 0 0 [swapper]
17 2 2 df8350c0 UN 0.0 0 0 [events/2]
18 2 3 df835500 UN 0.0 0 0 [events/3]
24 2 3 df836e80 UN 0.0 0 0 [xenwatch]
crash> bt 17
PID: 17 TASK: df8350c0 CPU: 2 COMMAND: "events/2"
#0 [df87be4c] schedule at c127d79f
#1 [df87bebc] schedule_timeout at c127daf4
#2 [df87befc] wait_for_common at c127d9fd
#3 [df87bf30] synchronize_sched at c10486b1
#4 [df87bf50] dev_deactivate at c11f8946
#5 [df87bf64] __linkwatch_run_queue at c11f270e
#6 [df87bf7c] linkwatch_event at c11f2760
#7 [df87bf80] worker_thread at c10473c3
#8 [df87bfc0] kthread at c1049eca
#9 [df87bfe4] kernel_thread_helper at c1008d25
crash> bt 18
PID: 18 TASK: df835500 CPU: 3 COMMAND: "events/3"
#0 [df87dec4] schedule at c127d79f
#1 [df87df34] __mutex_lock_common at c127df3a
#2 [df87df68] __mutex_lock_slowpath at c127df9c
#3 [df87df70] mutex_lock at c127e02d
#4 [df87df7c] linkwatch_event at c11f274d
#5 [df87df80] worker_thread at c10473c3
#6 [df87dfc0] kthread at c1049eca
#7 [df87dfe4] kernel_thread_helper at c1008d25
crash> bt 24
PID: 24 TASK: df836e80 CPU: 3 COMMAND: "xenwatch"
#0 [df8b3e88] schedule at c127d79f
#1 [df8b3ef8] __mutex_lock_common at c127df3a
#2 [df8b3f2c] __mutex_lock_slowpath at c127df9c
#3 [df8b3f34] mutex_lock at c127e02d
#4 [df8b3f40] netif_notify_peers at c11f8c71
#5 [df8b3f48] backend_changed at e0822f15
#6 [df8b3f9c] xenwatch_thread at c1196dd6
#7 [df8b3fc0] kthread at c1049eca
#8 [df8b3fe4] kernel_thread_helper at c1008d25
crash>
=> pids 24 and 18 are waiting for a mutex, probably rtnlmutex
=> pid 17 has acquired the rtnlmutex.
=> What is supposed wake pid 17 up?
6) We took another memory dump just to obvserve that the value of jiffies is not changing:
crash> print jiffies
$8 = 91395
crash> quit
$ crash -s --no_panic -e emacs vmlinux-2.6.32-5-686-bigmem squeeze32.2.dump
crash> print jiffies
$8 = 91395
However, "xm list -l" shows that the cpu usage time is slowly increasing.
7) after about 3 hours of waiting the domU magically recovered:
Nov 2 14:17:32 squeeze32 kernel: imklog 4.6.4, log source = /proc/kmsg started.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Reserving virtual address space above 0xf5800000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Initializing cgroup subsys cpuset
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Initializing cgroup subsys cpu
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Linux version 2.6.32-5-686-bigmem (Debian 2.6.32-26) (dannf@debian.org) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 SMP Tue Oct 19 15:03:03 UTC 2010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] KERNEL supported cpus:
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Intel GenuineIntel
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] AMD AuthenticAMD
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] NSC Geode by NSC
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Cyrix CyrixInstead
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Centaur CentaurHauls
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Transmeta GenuineTMx86
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Transmeta TransmetaCPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] UMC UMC UMC UMC
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] ACPI in unprivileged domain disabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] BIOS-provided physical RAM map:
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: 0000000000100000 - 0000000020000000 (usable)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMI not present or invalid.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] last_pfn = 0x20000 max_arch_pfn = 0x1000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] initial memory mapped : 0 - 033ff000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] init_memory_mapping: 0000000000000000-0000000020000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] NX (Execute Disable) protection: active
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 0000000000 - 0020000000 page 4k
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] kernel direct mapping tables up to 20000000 @ 100000-205000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] RAMDISK: 01620000 - 029f5000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 0MB HIGHMEM available.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 512MB LOWMEM available.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] mapped low ram: 0 - 20000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] low ram: 0 - 20000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] node 0 low ram: 00000000 - 20000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] node 0 bootmap 00002000 - 00006000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] (9 early reservations) ==> bootmem [0000000000 - 0020000000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #1 [0002a78000 - 0002a90000] XEN PAGETABLES ==> [0002a78000 - 0002a90000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #2 [0000001000 - 0000002000] EX TRAMPOLINE ==> [0000001000 - 0000002000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #3 [0000006000 - 0000007000] TRAMPOLINE ==> [0000006000 - 0000007000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #4 [0001000000 - 00014fa3f4] TEXT DATA BSS ==> [0001000000 - 00014fa3f4]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #5 [0001620000 - 00029f5000] RAMDISK ==> [0001620000 - 00029f5000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #6 [00029f5000 - 0002a78000] XEN START INFO ==> [00029f5000 - 0002a78000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #7 [0000100000 - 00001e9000] PGTABLE ==> [0000100000 - 00001e9000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #8 [0000002000 - 0000006000] BOOTMAP ==> [0000002000 - 0000006000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Zone PFN ranges:
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMA 0x00000000 -> 0x00001000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Normal 0x00001000 -> 0x00020000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] HighMem 0x00020000 -> 0x00020000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Movable zone start PFN for each node
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] early_node_map[2] active PFN ranges
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 0: 0x00000000 -> 0x000000a0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 0: 0x00000100 -> 0x00020000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] On node 0 totalpages: 130976
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] free_area_init_node: node 0, pgdat c13cff40, node_mem_map c2a90000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMA zone: 32 pages used for memmap
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMA zone: 0 pages reserved
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMA zone: 3968 pages, LIFO batch:0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Normal zone: 992 pages used for memmap
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Normal zone: 125984 pages, LIFO batch:31
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Using APIC driver default
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] SFI: Simple Firmware Interface v0.7 http://simplefirmware.org
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Local APIC disabled by BIOS -- you can enable it with "lapic"
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] APIC: disable apic facility
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] nr_irqs_gsi: 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Allocating PCI resources starting at 20000000 (gap: 20000000:e0000000)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Booting paravirtualized kernel on Xen
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen version: 4.0.1 (preserve-AD)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:4 nr_node_ids:1
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] PERCPU: Embedded 15 pages/cpu @c2e94000 s37656 r0 d23784 u65536
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] pcpu-alloc: s37656 r0 d23784 u65536 alloc=16*4096
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] trying to map vcpu_info 0 at c2e95010, mfn 20d098, offset 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] cpu 0 using vcpu_info at c2e95010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] trying to map vcpu_info 1 at c2ea5010, mfn 20d088, offset 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] cpu 1 using vcpu_info at c2ea5010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] trying to map vcpu_info 2 at c2eb5010, mfn 20d078, offset 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] cpu 2 using vcpu_info at c2eb5010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] trying to map vcpu_info 3 at c2ec5010, mfn 20d068, offset 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] cpu 3 using vcpu_info at c2ec5010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: using vcpu_info placement
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129952
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Kernel command line: root=/dev/xvda ro console=hvc0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Enabling fast FPU save and restore... done.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Enabling unmasked SIMD FPU exception support... done.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Initializing CPU#0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Initializing HighMem for node 0 (00000000:00000000)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Memory: 491968k/524288k available (2569k kernel code, 31748k reserved, 1374k data, 388k init, 0k highmem)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] virtual kernel memory layout:
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] fixmap : 0xf5556000 - 0xf57ff000 (2724 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] pkmap : 0xf5000000 - 0xf5200000 (2048 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] vmalloc : 0xe0800000 - 0xf4ffe000 ( 327 MB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] lowmem : 0xc0000000 - 0xe0000000 ( 512 MB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] .init : 0xc13da000 - 0xc143b000 ( 388 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] .data : 0xc128244d - 0xc13d9cf0 (1374 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] .text : 0xc1000000 - 0xc128244d (2569 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Hierarchical RCU implementation.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] NR_IRQS:1280
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Console: colour dummy device 80x25
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] console [tty0] enabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] console [hvc0] enabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: using vcpuop timer interface
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] installing Xen timer for CPU 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Detected 2394.036 MHz processor.
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 4788.07 BogoMIPS (lpj=9576144)
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Security Framework initialized
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] SELinux: Disabled at boot.
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Mount-cache hash table entries: 512
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys ns
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys cpuacct
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys devices
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys freezer
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys net_cls
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L2 cache: 4096K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Physical Processor ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Processor Core ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Performance Events: unsupported p6 CPU model 15 no PMU driver, software events only.
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] SMP alternatives: switching to UP code
Nov 2 14:17:32 squeeze32 kernel: [ 0.004399] installing Xen timer for CPU 1
Nov 2 14:17:32 squeeze32 kernel: [ 0.004431] SMP alternatives: switching to SMP code
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing CPU#1
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L2 cache: 4096K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Physical Processor ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Processor Core ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.008952] installing Xen timer for CPU 2
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing CPU#2
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L2 cache: 4096K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Physical Processor ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Processor Core ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.009241] installing Xen timer for CPU 3
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing CPU#3
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L2 cache: 4096K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Physical Processor ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Processor Core ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.009393] Brought up 4 CPUs
Nov 2 14:17:32 squeeze32 kernel: [ 0.009447] CPU0 attaching sched-domain:
Nov 2 14:17:32 squeeze32 kernel: [ 0.009454] domain 0: span 0-3 level CPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.009463] groups: 0 1 2 3
Nov 2 14:17:32 squeeze32 kernel: [ 0.009487] CPU1 attaching sched-domain:
Nov 2 14:17:32 squeeze32 kernel: [ 0.009493] domain 0: span 0-3 level CPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.009501] groups: 1 2 3 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.009524] CPU2 attaching sched-domain:
Nov 2 14:17:32 squeeze32 kernel: [ 0.009530] domain 0: span 0-3 level CPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.009538] groups: 2 3 0 1
Nov 2 14:17:32 squeeze32 kernel: [ 0.009561] CPU3 attaching sched-domain:
Nov 2 14:17:32 squeeze32 kernel: [ 0.009566] domain 0: span 0-3 level CPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.009575] groups: 3 0 1 2
Nov 2 14:17:32 squeeze32 kernel: [ 0.009738] devtmpfs: initialized
Nov 2 14:17:32 squeeze32 kernel: [ 0.009738] Grant table initialized
Nov 2 14:17:32 squeeze32 kernel: [ 0.009738] regulator: core version 0.5
Nov 2 14:17:32 squeeze32 kernel: [ 0.009738] NET: Registered protocol family 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.012050] PCI: Fatal: No config space access function found
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] bio: create slab <bio-0> at 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] ACPI: Interpreter disabled.
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] xen_balloon: Initialising balloon driver.
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] vgaarb: loaded
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] PCI: System does not support PCI
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] PCI: System does not support PCI
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] Switching to clocksource xen
Nov 2 14:17:32 squeeze32 kernel: [ 0.019527] pnp: PnP ACPI: disabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.019540] PnPBIOS: Disabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.020189] NET: Registered protocol family 2
Nov 2 14:17:32 squeeze32 kernel: [ 0.020303] IP route cache hash table entries: 4096 (order: 2, 16384 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.119927] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.120005] TCP bind hash table entries: 16384 (order: 5, 131072 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.120072] TCP: Hash tables configured (established 16384 bind 16384)
Nov 2 14:17:32 squeeze32 kernel: [ 0.120086] TCP reno registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.120204] NET: Registered protocol family 1
Nov 2 14:17:32 squeeze32 kernel: [ 0.120318] Unpacking initramfs...
Nov 2 14:17:32 squeeze32 kernel: [ 0.153610] Freeing initrd memory: 20308k freed
Nov 2 14:17:32 squeeze32 kernel: [ 0.162590] platform rtc_cmos: registered platform RTC device (no PNP device found)
Nov 2 14:17:32 squeeze32 kernel: [ 0.163047] audit: initializing netlink socket (disabled)
Nov 2 14:17:32 squeeze32 kernel: [ 0.163071] type=2000 audit(1288700248.679:1): initialized
Nov 2 14:17:32 squeeze32 kernel: [ 0.166293] HugeTLB registered 2 MB page size, pre-allocated 0 pages
Nov 2 14:17:32 squeeze32 kernel: [ 0.168396] VFS: Disk quotas dquot_6.5.2
Nov 2 14:17:32 squeeze32 kernel: [ 0.168462] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.168598] msgmni has been set to 1000
Nov 2 14:17:32 squeeze32 kernel: [ 0.169229] alg: No test for stdrng (krng)
Nov 2 14:17:32 squeeze32 kernel: [ 0.169311] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
Nov 2 14:17:32 squeeze32 kernel: [ 0.169323] io scheduler noop registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.169330] io scheduler anticipatory registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.169338] io scheduler deadline registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.169389] io scheduler cfq registered (default)
Nov 2 14:17:32 squeeze32 kernel: [ 0.169586] isapnp: Write Data Register 0xa79 already used
Nov 2 14:17:32 squeeze32 kernel: [ 0.173466] Linux agpgart interface v0.103
Nov 2 14:17:32 squeeze32 kernel: [ 0.173710] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.174084] PNP: No PS/2 controller found. Probing ports directly.
Nov 2 14:17:32 squeeze32 kernel: [ 0.174919] i8042.c: No controller found.
Nov 2 14:17:32 squeeze32 kernel: [ 0.174994] mice: PS/2 mouse device common for all mice
Nov 2 14:17:32 squeeze32 kernel: [ 0.175058] rtc_cmos: probe of rtc_cmos failed with error -16
Nov 2 14:17:32 squeeze32 kernel: [ 0.175096] cpuidle: using governor ladder
Nov 2 14:17:32 squeeze32 kernel: [ 0.175103] cpuidle: using governor menu
Nov 2 14:17:32 squeeze32 kernel: [ 0.175113] No iBFT detected.
Nov 2 14:17:32 squeeze32 kernel: [ 0.175391] TCP cubic registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.175574] NET: Registered protocol family 10
Nov 2 14:17:32 squeeze32 kernel: [ 0.175973] lo: Disabled Privacy Extensions
Nov 2 14:17:32 squeeze32 kernel: [ 0.176215] Mobile IPv6
Nov 2 14:17:32 squeeze32 kernel: [ 0.176223] NET: Registered protocol family 17
Nov 2 14:17:32 squeeze32 kernel: [ 0.176242] Using IPI No-Shortcut mode
Nov 2 14:17:32 squeeze32 kernel: [ 0.176326] PM: Resume from disk failed.
Nov 2 14:17:32 squeeze32 kernel: [ 0.176342] registered taskstats version 1
Nov 2 14:17:32 squeeze32 kernel: [ 0.176376] XENBUS: Device with no driver: device/vbd/51712
Nov 2 14:17:32 squeeze32 kernel: [ 0.176383] XENBUS: Device with no driver: device/vif/0
Nov 2 14:17:32 squeeze32 kernel: [ 0.176391] XENBUS: Device with no driver: device/console/0
Nov 2 14:17:32 squeeze32 kernel: [ 0.176412] /tmp/buildd/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Nov 2 14:17:32 squeeze32 kernel: [ 0.176445] Initalizing network drop monitor service
Nov 2 14:17:32 squeeze32 kernel: [ 0.176491] Freeing unused kernel memory: 388k freed
Nov 2 14:17:32 squeeze32 kernel: [ 0.177527] Write protecting the kernel text: 2572k
Nov 2 14:17:32 squeeze32 kernel: [ 0.177839] Write protecting the kernel read-only data: 924k
Nov 2 14:17:32 squeeze32 kernel: [ 0.218177] udev[77]: starting version 163
Nov 2 14:17:32 squeeze32 kernel: [ 0.240302] Initialising Xen virtual ethernet driver.
Nov 2 14:17:32 squeeze32 kernel: [ 0.248886] blkfront: xvda: barriers enabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.249192] xvda: unknown partition table
Nov 2 14:17:32 squeeze32 kernel: [ 0.431926] kjournald starting. Commit interval 5 seconds
Nov 2 14:17:32 squeeze32 kernel: [ 0.431944] EXT3-fs: mounted filesystem with ordered data mode.
Nov 2 14:17:32 squeeze32 kernel: [ 1.355631] udev[158]: starting version 163
Nov 2 14:17:32 squeeze32 kernel: [ 1.707941] input: PC Speaker as /devices/platform/pcspkr/input/input0
Nov 2 14:17:32 squeeze32 kernel: [ 1.849344] Error: Driver 'pcspkr' is already registered, aborting...
Nov 2 14:17:32 squeeze32 kernel: [ 2.153273] EXT3 FS on xvda, internal journal
Nov 2 14:17:32 squeeze32 kernel: [ 2.250745] loop: module loaded
Nov 2 14:17:42 squeeze32 kernel: [ 13.692056] eth0: no IPv6 routers present
Nov 2 14:18:58 squeeze32 kernel: [ 89.571723] suspending xenstore...
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] trying to map vcpu_info 0 at c2e95010, mfn 13188a, offset 16
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] cpu 0 using vcpu_info at c2e95010
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] trying to map vcpu_info 1 at c2ea5010, mfn 13187a, offset 16
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] cpu 1 using vcpu_info at c2ea5010
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] trying to map vcpu_info 2 at c2eb5010, mfn 13186a, offset 16
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] cpu 2 using vcpu_info at c2eb5010
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] trying to map vcpu_info 3 at c2ec5010, mfn 13185a, offset 16
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] cpu 3 using vcpu_info at c2ec5010
Nov 2 14:21:21 squeeze32 kernel: [ 191.777200] suspending xenstore...
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] trying to map vcpu_info 0 at c2e95010, mfn 13188a, offset 16
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] cpu 0 using vcpu_info at c2e95010
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] trying to map vcpu_info 1 at c2ea5010, mfn 13187a, offset 16
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] cpu 1 using vcpu_info at c2ea5010
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] trying to map vcpu_info 2 at c2eb5010, mfn 13186a, offset 16
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] cpu 2 using vcpu_info at c2eb5010
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] trying to map vcpu_info 3 at c2ec5010, mfn 13185a, offset 16
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] cpu 3 using vcpu_info at c2ec5010
Nov 2 14:34:27 squeeze32 kernel: [ 665.577176] suspending xenstore...
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] trying to map vcpu_info 0 at c2e95010, mfn 131889, offset 16
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] cpu 0 using vcpu_info at c2e95010
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] trying to map vcpu_info 1 at c2ea5010, mfn 131879, offset 16
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] cpu 1 using vcpu_info at c2ea5010
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] trying to map vcpu_info 2 at c2eb5010, mfn 131869, offset 16
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] cpu 2 using vcpu_info at c2eb5010
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] trying to map vcpu_info 3 at c2ec5010, mfn 131859, offset 16
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] cpu 3 using vcpu_info at c2ec5010
Nov 2 17:44:23 squeeze32 kernel: [ 665.597226] SysRq : HELP : loglevel(0-9) reBoot Crash terminate-all-tasks(E) memory-full-oom-kill(F) kill-all-tasks(I) thaw-filesystems(J) saK show-backtrace-all-active-cpus(L) show-memory-usage(M) nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw Sync show-task-states(T) Unmount show-blocked-tasks(W) dump-ftrace-buffer(Z)
=> This 3 hours and 10 minutes seems to correspond to the uptime that
dom0 had before its reboot:
overlord2:~$ last reboot
reboot system boot 2.6.32-5-xen-amd Tue Nov 2 14:36 - 09:34 (18:58)
reboot system boot 2.6.32-5-xen-amd Tue Nov 2 11:26 - 14:34 (03:07)
reboot system boot 2.6.32-5-xen-amd Tue Nov 2 09:37 - 11:24 (01:46)
reboot system boot 2.6.32-5-xen-amd Tue Nov 2 09:34 - 09:36 (00:02)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 16:44 - 09:32 (16:47)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 16:39 - 16:43 (00:03)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 16:25 - 16:29 (00:04)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 15:47 - 16:29 (00:42)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 15:27 - 15:44 (00:17)
=> Theory: some counter is reset when dom0 is rebooted. domU becomes
very upset when the counter suddenly jumps backwards and recovers only
when the counter has finally reached its previous value?
Reply to: