--- Begin Message ---
Package: linux-2.6
Version: 2.6.32-23
Severity: grave
Tags: upstream
Justification: renders package unusable
-- Running NFS inside of a DomU when connected to Dom0 causes kernel
panic within a few minutes. Does not matter which DomU the NFS is
connected from. Does not happen when connecting to NFS outside of the
Xen enviroment.
Notes I've made while diagnosing this issue:
1. Able to run multiple Xen DomUs (8 in total)
2. MX1 has a NFS connection to Dom0 (/home) for mail delivery.
3. I can run MX1 without NFS with no issues, No oops or any issues
4. within a few mins of mounting NFS on MX1 Dom0 crashes
5. Clock-sources are a mixture of XEN and TSC (Xen is default, moved 2
DomUs to TSC as a troubleshooting step (aaa and MX1)
6. Mounted /home on a different DomU as a test..
7. Leaving the NFS mount on a different DomU did not cause Panic,
however, no traffic happened left it for a few hours and no panic.
8. Started moving files between NFS and local DomU drive and Panic happened.
9. Coping large amounts of files within the DomU (not over NFS) did not
seem to cause Oops.
10. Created NFS share on desktop which is not part of the Xen server to
see if it causes a panic (Identifies if this is a Dom0 or DomU issue)
NOTE: Dom0 has /home which the desktop can access perfectly fine.
11. Moving files from Desktop NFS is ok.
Recap:
DomU <-> DomU: Ok
DomU <-> Desktop NFS: OK
DomU <-> Dom0: Oops
Desktop <-> Dom0: OK
-- Package-specific info:
** Version:
Linux version 2.6.32-5-xen-686 (Debian 2.6.32-23) (dannf@debian.org)
(gcc version 4.3.5 (Debian 4.3.5-3) ) #1 SMP Sat Sep 18 06:00:14 UTC 2010
** Command line:
placeholder root=UUID=6b304f06-c37f-480f-af92-478443531682 ro quiet
** Not tainted
** Kernel log:
[ 157.841231] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.841242] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.841253] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.841265] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872702] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872718] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872728] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872738] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872748] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872759] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872769] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872778] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872788] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872798] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872808] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872818] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.872828] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884349] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884372] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884388] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884405] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884420] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884437] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884455] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884471] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884488] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884506] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884521] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884537] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884555] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 157.884570] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 161.642902] blkback: ring-ref 8, event-channel 8, protocol 1 (x86_32-abi)
[ 161.935808] blkback: ring-ref 9, event-channel 9, protocol 1 (x86_32-abi)
[ 168.717532] vif7.0: no IPv6 routers present
[ 172.787784] device vif8.0 entered promiscuous mode
[ 172.797787] eth0: port 9(vif8.0) entering forwarding state
[ 172.848319] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848343] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848359] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848374] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848388] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848402] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848420] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848437] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848453] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848468] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848482] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848496] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848511] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.848526] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860830] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860843] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860854] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860864] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860874] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860884] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860894] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860904] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860914] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860924] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860934] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860944] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860954] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860963] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.860974] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874077] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874091] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874101] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874111] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874121] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874131] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874141] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874151] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874161] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874170] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874180] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874190] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874200] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874210] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.874220] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.887977] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.887995] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888088] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888119] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888134] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888152] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888171] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888186] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888202] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888222] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888236] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888252] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888270] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888286] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888301] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 172.888321] physdev match: using --physdev-out in the OUTPUT, FORWARD
and POSTROUTING chains for non-bridged traffic is not supported anymore.
[ 177.711666] blkback: ring-ref 8, event-channel 8, protocol 1 (x86_32-abi)
[ 178.054806] blkback: ring-ref 9, event-channel 9, protocol 1 (x86_32-abi)
[ 181.495854] Switching to clocksource tsc
[ 182.880089] vif8.0: no IPv6 routers present
** Model information
not available
** Loaded modules:
Module Size Used by
xt_tcpudp 1743 8
nf_conntrack_ipv4 7597 16
nf_defrag_ipv4 779 1 nf_conntrack_ipv4
xt_state 927 16
nf_conntrack 38079 2 nf_conntrack_ipv4,xt_state
xt_physdev 1228 32
iptable_filter 1790 1
ip_tables 7690 1 iptable_filter
x_tables 8327 4 xt_tcpudp,xt_state,xt_physdev,ip_tables
bridge 32999 0
stp 996 1 bridge
xen_evtchn 3471 9
xenfs 7248 1
nfsd 193247 13
exportfs 2618 1 nfsd
nfs 206894 0
lockd 49433 2 nfsd,nfs
fscache 24654 1 nfs
nfs_acl 1695 2 nfsd,nfs
auth_rpcgss 25240 2 nfsd,nfs
sunrpc 134572 14 nfsd,nfs,lockd,nfs_acl,auth_rpcgss
loop 9761 32
snd_pcm 47402 0
snd_timer 12258 1 snd_pcm
pl2303 11624 0
dcdbas 3940 0
snd 34363 2 snd_pcm,snd_timer
usbserial 21843 1 pl2303
soundcore 3450 1 snd
snd_page_alloc 5109 1 snd_pcm
psmouse 44653 0
pcspkr 1207 0
serio_raw 2916 0
shpchp 21264 0
evdev 5609 2
i2c_piix4 7124 0
pci_hotplug 18065 1 shpchp
i2c_core 12696 1 i2c_piix4
button 3598 0
processor 31041 0
acpi_processor 3623 1 processor,[permanent]
ext3 94196 2
jbd 32161 1 ext3
mbcache 3762 1 ext3
sr_mod 10770 0
cdrom 26487 1 sr_mod
ata_generic 2019 0
sg 15972 0
sd_mod 25865 5
crc_t10dif 1012 1 sd_mod
megaraid_mbox 21022 0
pata_serverworks 3969 0
ohci_hcd 17068 0
megaraid_mm 5451 1 megaraid_mbox
libata 116021 2 ata_generic,pata_serverworks
qla1280 16957 0
tg3 88633 0
floppy 40931 0
aacraid 51565 3
libphy 11638 1 tg3
ehci_hcd 28255 0
scsi_mod 101397 7
sr_mod,sg,sd_mod,megaraid_mbox,libata,qla1280,aacraid
usbcore 98725 5 pl2303,usbserial,ohci_hcd,ehci_hcd
nls_base 4541 1 usbcore
thermal 9206 0
thermal_sys 9378 2 processor,thermal
** Network interface configuration:
auto lo
iface lo inet loopback
allow-hotplug eth0
iface eth0 inet static
address 10.0.0.20
netmask 255.255.255.0
broadcast 10.0.0.255
gateway 10.0.0.1
dns-domain lab.ostlabs.com
dns-nameservers 10.0.0.1
** Network status:
*** IP interfaces and addresses:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: peth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq
state UP qlen 1000
link/ether 00:06:5b:f7:eb:c3 brd ff:ff:ff:ff:ff:ff
inet6 fe80::206:5bff:fef7:ebc3/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:06:5b:f7:eb:c4 brd ff:ff:ff:ff:ff:ff
4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
link/ether 00:06:5b:f7:eb:c3 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.20/24 brd 10.0.0.255 scope global eth0
inet6 fe80::206:5bff:fef7:ebc3/64 scope link
valid_lft forever preferred_lft forever
5: vif1.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UNKNOWN qlen 32
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcff:ffff:feff:ffff/64 scope link
valid_lft forever preferred_lft forever
6: vif2.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UNKNOWN qlen 32
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcff:ffff:feff:ffff/64 scope link
valid_lft forever preferred_lft forever
7: vif3.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UNKNOWN qlen 32
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcff:ffff:feff:ffff/64 scope link
valid_lft forever preferred_lft forever
8: vif4.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UNKNOWN qlen 32
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcff:ffff:feff:ffff/64 scope link
valid_lft forever preferred_lft forever
9: vif5.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UNKNOWN qlen 32
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcff:ffff:feff:ffff/64 scope link
valid_lft forever preferred_lft forever
10: vif6.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UNKNOWN qlen 32
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcff:ffff:feff:ffff/64 scope link
valid_lft forever preferred_lft forever
11: vif7.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UNKNOWN qlen 32
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcff:ffff:feff:ffff/64 scope link
valid_lft forever preferred_lft forever
12: vif8.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UNKNOWN qlen 32
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcff:ffff:feff:ffff/64 scope link
valid_lft forever preferred_lft forever
*** Device statistics:
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed
multicast|bytes packets errs drop fifo colls carrier compressed
lo: 12990 151 0 0 0 0 0 0
12990 151 0 0 0 0 0 0
peth0:33668865 52529 0 0 0 0 0 1400
40069263 51090 0 0 0 0 0 0
eth1: 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
eth0:32154816 49419 0 0 0 0 0 2082
38628135 34224 0 0 0 0 0 0
vif1.0: 320314 3264 0 0 0 0 0 0
742272 6508 0 13 0 0 0 0
vif2.0: 547294 6543 0 0 0 0 0 0
1059726 7321 0 9 0 0 0 0
vif3.0: 606762 1869 0 0 0 0 0 0
633826 4139 0 13 0 0 0 0
vif4.0: 67861 381 0 0 0 0 0 0
336287 2440 0 13 0 0 0 0
vif5.0: 118042 1447 0 0 0 0 0 0
466361 3261 0 15 0 0 0 0
vif6.0: 346370 2546 0 0 0 0 0 0
956549 4325 0 16 0 0 0 0
vif7.0: 18773 316 0 0 0 0 0 0
303830 2198 0 10 0 0 0 0
vif8.0: 804 13 0 0 0 0 0 0
271472 1921 0 16 0 0 0 0
*** Protocol statistics:
Ip:
48197 total packets received
38 with invalid addresses
0 forwarded
0 incoming packets discarded
48009 incoming packets delivered
34936 requests sent out
13 dropped because of missing route
Icmp:
6 ICMP messages received
0 input ICMP message failed.
ICMP input histogram:
destination unreachable: 6
73 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 73
IcmpMsg:
InType3: 6
OutType3: 73
Tcp:
11 active connections openings
38 passive connection openings
6 failed connection attempts
0 connection resets received
3 connections established
47193 segments received
34001 segments send out
25 segments retransmited
0 bad segments received.
8 resets sent
Udp:
416 packets received
67 packets to unknown port received.
0 packet receive errors
841 packets sent
UdpLite:
TcpExt:
3 TCP sockets finished time wait in fast timer
109 delayed acks sent
5 delayed acks further delayed because of locked socket
1 packets directly queued to recvmsg prequeue.
26117 packet headers predicted
940 acknowledgments not containing data payload received
24570 predicted acknowledgments
2 times recovered from packet loss by selective acknowledgements
76 TCP data loss events
TCPLostRetransmit: 4
25 fast retransmits
TCPSackShifted: 74
TCPSackMerged: 135
TCPSackShiftFallback: 20
IpExt:
InBcastPkts: 602
OutBcastPkts: 51
InOctets: 32108796
OutOctets: 38224368
InBcastOctets: 159844
OutBcastOctets: 7380
*** Device features:
eth0: 0x11823
eth1: 0x9a3
lo: 0x13865
peth0: 0x9a3
vif1.0: 0x10803
vif2.0: 0x10803
vif3.0: 0x10803
vif4.0: 0x10803
vif5.0: 0x10803
vif6.0: 0x10803
vif7.0: 0x10803
vif8.0: 0x10803
** PCI devices:
00:00.0 Host bridge [0600]: Broadcom CMIC-WS Host Bridge (GC-LE chipset)
[1166:0012] (rev 13)
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
00:00.1 Host bridge [0600]: Broadcom CMIC-WS Host Bridge (GC-LE chipset)
[1166:0012]
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
00:00.2 Host bridge [0600]: Broadcom CMIC-LE [1166:0000]
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
00:04.0 Unassigned class [ff00]: Dell Embedded Remote Access or ERA/O
[1028:000c]
Subsystem: Dell Embedded Remote Access or ERA/O [1028:000c]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 11
Region 0: Memory at feb80000 (32-bit, prefetchable) [size=4K]
Region 1: I/O ports at ecf8 [size=8]
Region 2: I/O ports at ece8 [size=8]
Expansion ROM at fe000000 [disabled] [size=64K]
Capabilities: <access denied>
00:04.1 Unassigned class [ff00]: Dell Remote Access Card III [1028:0008]
Subsystem: Dell Remote Access Card III [1028:0008]
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin B routed to IRQ 23
Region 0: Memory at fe102000 (32-bit, non-prefetchable) [size=4K]
Region 1: I/O ports at ec80 [size=64]
Region 2: Memory at feb00000 (32-bit, prefetchable) [size=512K]
Capabilities: <access denied>
Kernel driver in use: serial
00:04.2 Unassigned class [ff00]: Dell Embedded Remote Access: BMC/SMIC
device [1028:000d]
Subsystem: Dell Embedded Remote Access: BMC/SMIC device [1028:000d]
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin C routed to IRQ 7
Region 0: I/O ports at ecf4 [size=4]
Capabilities: <access denied>
00:0e.0 VGA compatible controller [0300]: ATI Technologies Inc Rage XL
[1002:4752] (rev 27) (prog-if 00 [VGA controller])
Subsystem: Dell Device [1028:0121]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop+ ParErr-
Stepping+ SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32 (2000ns min), Cache Line Size: 64 bytes
Region 0: Memory at fd000000 (32-bit, non-prefetchable) [size=16M]
Region 1: I/O ports at e800 [size=256]
Region 2: Memory at fe101000 (32-bit, non-prefetchable) [size=4K]
[virtual] Expansion ROM at e8000000 [disabled] [size=128K]
Capabilities: <access denied>
00:0f.0 Host bridge [0600]: Broadcom CSB5 South Bridge [1166:0201] (rev 93)
Subsystem: Broadcom CSB5 South Bridge [1166:0201]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Latency: 32
Kernel driver in use: piix4_smbus
00:0f.1 IDE interface [0101]: Broadcom CSB5 IDE Controller [1166:0212]
(rev 93) (prog-if 82 [Master PriP])
Subsystem: Broadcom CSB5 IDE Controller [1166:0212]
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64
Region 0: I/O ports at 01f0 [size=8]
Region 1: I/O ports at 03f4 [size=1]
Region 2: I/O ports at 0170 [size=8]
Region 3: I/O ports at 0374 [size=1]
Region 4: I/O ports at 08b0 [size=16]
Kernel driver in use: pata_serverworks
00:0f.2 USB Controller [0c03]: Broadcom OSB4/CSB5 OHCI USB Controller
[1166:0220] (rev 05) (prog-if 10 [OHCI])
Subsystem: Broadcom OSB4/CSB5 OHCI USB Controller [1166:0220]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32 (20000ns max)
Interrupt: pin A routed to IRQ 11
Region 0: Memory at fe100000 (32-bit, non-prefetchable) [size=4K]
Kernel driver in use: ohci_hcd
00:0f.3 ISA bridge [0601]: Broadcom CSB5 LPC bridge [1166:0225]
Subsystem: Broadcom Dell PowerEdge 1650/1750 [1166:0230]
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
00:10.0 Host bridge [0600]: Broadcom CIOB-X2 PCI-X I/O Bridge
[1166:0101] (rev 03)
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Capabilities: <access denied>
00:10.2 Host bridge [0600]: Broadcom CIOB-X2 PCI-X I/O Bridge
[1166:0101] (rev 03)
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Capabilities: <access denied>
00:11.0 Host bridge [0600]: Broadcom CIOB-X2 PCI-X I/O Bridge
[1166:0101] (rev 05)
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Capabilities: <access denied>
00:11.2 Host bridge [0600]: Broadcom CIOB-X2 PCI-X I/O Bridge
[1166:0101] (rev 05)
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Capabilities: <access denied>
02:06.0 PCI bridge [0604]: Intel Corporation 21154 PCI-to-PCI Bridge
[8086:b154] (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32
Bus: primary=02, secondary=03, subordinate=04, sec-latency=32
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: fcd00000-fcffffff
Prefetchable memory behind bridge: 00000000f0000000-00000000f7ffffff
Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: <access denied>
03:00.0 PCI bridge [0604]: Intel Corporation 21154 PCI-to-PCI Bridge
[8086:b154] (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32
Bus: primary=03, secondary=04, subordinate=04, sec-latency=32
Memory behind bridge: fcf00000-fcffffff
Prefetchable memory behind bridge: 00000000f0000000-00000000f7ffffff
Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: <access denied>
03:01.0 SCSI storage controller [0100]: QLogic Corp. ISP12160 Dual
Channel Ultra3 SCSI Processor [1077:1216] (rev 06)
Subsystem: American Megatrends Inc. QLA12160 on AMI MegaRAID
[101e:8493]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32 (16000ns min), Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 11
Region 0: I/O ports at dc00 [size=256]
Region 1: Memory at fcdff000 (32-bit, non-prefetchable) [size=4K]
Expansion ROM at fce00000 [disabled] [size=128K]
Capabilities: <access denied>
04:00.0 RAID bus controller [0104]: American Megatrends Inc. MegaRAID
[101e:1960] (rev 20)
Subsystem: Dell PowerEdge RAID Controller 3/DC [1028:0493]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Latency: 32, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 24
Region 0: Memory at f0000000 (32-bit, prefetchable) [size=128M]
Expansion ROM at fcf00000 [disabled] [size=128K]
Capabilities: <access denied>
Kernel driver in use: megaraid
05:06.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme
BCM5701 Gigabit Ethernet [14e4:1645] (rev 15)
Subsystem: Dell Broadcom BCM5701 1000Base-T [1028:0121]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64 (16000ns min), Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 28
Region 0: Memory at eff10000 (64-bit, non-prefetchable) [size=64K]
Capabilities: <access denied>
Kernel driver in use: tg3
05:08.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme
BCM5701 Gigabit Ethernet [14e4:1645] (rev 15)
Subsystem: Dell Broadcom BCM5701 1000Base-T [1028:0121]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64 (16000ns min), Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 29
Region 0: Memory at eff00000 (64-bit, non-prefetchable) [size=64K]
Capabilities: <access denied>
Kernel driver in use: tg3
06:08.0 PCI bridge [0604]: Intel Corporation 80303 I/O Processor
PCI-to-PCI Bridge [8086:0309] (rev 01) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32, Cache Line Size: 64 bytes
Bus: primary=06, secondary=07, subordinate=07, sec-latency=32
Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: <access denied>
06:08.1 RAID bus controller [0104]: Dell PowerEdge Expandable RAID
Controller 3/Di [1028:000a] (rev 01)
Subsystem: Dell PERC 3/DiB [Boxster] [1028:0121]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Latency: 32, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 30
Region 0: Memory at e0000000 (32-bit, prefetchable) [size=128M]
Expansion ROM at efd00000 [disabled] [size=64K]
Capabilities: <access denied>
Kernel driver in use: aacraid
** USB devices:
Bus 001 Device 002: ID 067b:aaa0 Prolific Technology, Inc. Prolific Pharos
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
-- System Information:
Debian Release: squeeze/sid
APT prefers testing
APT policy: (500, 'testing')
Architecture: i386 (i686)
Kernel: Linux 2.6.32-5-xen-686 (SMP w/4 CPU cores)
Locale: LANG=en_CA.UTF-8, LC_CTYPE=en_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages linux-image-2.6.32-5-xen-686 depends on:
ii debconf [debconf-2.0] 1.5.35 Debian configuration
management sy
ii initramfs-tools 0.98.4 tools for generating an
initramfs
ii linux-base 2.6.32-23 Linux image base package
ii module-init-tools 3.12-1 tools for managing Linux
kernel mo
Versions of packages linux-image-2.6.32-5-xen-686 recommends:
ii firmware-linux-free 2.6.32-23 Binary firmware for various
driver
ii libc6-xen 2.11.2-6 Embedded GNU C Library:
Shared lib
Versions of packages linux-image-2.6.32-5-xen-686 suggests:
pn grub <none> (no description available)
pn linux-doc-2.6.32 <none> (no description available)
Versions of packages linux-image-2.6.32-5-xen-686 is related to:
pn firmware-bnx2 <none> (no description available)
pn firmware-bnx2x <none> (no description available)
pn firmware-ipw2x00 <none> (no description available)
pn firmware-ivtv <none> (no description available)
pn firmware-iwlwifi <none> (no description available)
pn firmware-linux <none> (no description available)
pn firmware-linux-nonfree <none> (no description available)
pn firmware-qlogic <none> (no description available)
pn firmware-ralink <none> (no description available)
ii xen-hypervisor-4.0-i386 [xen- 4.0.1-1 The Xen Hypervisor on i386
-- debconf information:
linux-image-2.6.32-5-xen-686/postinst/depmod-error-initrd-2.6.32-5-xen-686:
false
linux-image-2.6.32-5-xen-686/postinst/ignoring-do-bootloader-2.6.32-5-xen-686:
linux-image-2.6.32-5-xen-686/prerm/removing-running-kernel-2.6.32-5-xen-686:
true
linux-image-2.6.32-5-xen-686/postinst/missing-firmware-2.6.32-5-xen-686:
------ netconsole Trace:
Oct 4 09:45:31 10.0.0.20 kernel: [173927.281586] netconsole: local port
6666
Oct 4 09:45:31 10.0.0.20 kernel: [173927.281599] netconsole: local IP
10.0.0.20
Oct 4 09:45:31 10.0.0.20 kernel: [173927.281605] netconsole: interface eth0
Oct 4 09:45:31 10.0.0.20 kernel: [173927.281612] netconsole: remote
port 514
Oct 4 09:45:31 10.0.0.20 kernel: [173927.281619] netconsole: remote IP
10.0.0.100
Oct 4 09:45:31 10.0.0.20 kernel: [173927.281627] netconsole: remote
ethernet address 00:1a:70:11:c5:ff
Oct 4 09:45:47 10.0.0.20 kernel: [173943.595807] netconsole: local port
6666
Oct 4 09:45:47 10.0.0.20 kernel: [173943.595820] netconsole: local IP
10.0.0.20
Oct 4 09:45:47 10.0.0.20 kernel: [173943.595830] netconsole: interface
peth0
Oct 4 09:45:47 10.0.0.20 kernel: [173943.595839] netconsole: remote
port 514
Oct 4 09:45:47 10.0.0.20 kernel: [173943.595848] netconsole: remote IP
10.0.0.100
Oct 4 09:45:47 10.0.0.20 kernel: [173943.595859] netconsole: remote
ethernet address 00:1a:70:11:c5:ff
Oct 4 09:45:47 10.0.0.20 kernel: [173943.605629] console [netcon0] enabled
Oct 4 09:45:47 10.0.0.20 kernel: [173943.605636] netconsole: network
logging started
Oct 4 09:52:29 10.0.0.20 [174345.332640] ------------[ cut here
]------------
Oct 4 09:52:29 10.0.0.20 [174345.332716] kernel BUG at
/build/buildd-linux-2.6_2.6.32-23-i386-x1D1UQ/linux-2.6-2.6.32/debian/build/source_i386_xen/drivers/xen/grant-table.c:584!
Oct 4 09:52:29 10.0.0.20 [174345.332805] invalid opcode: 0000 [#1]
Oct 4 09:52:29 10.0.0.20 SMP
Oct 4 09:52:29 10.0.0.20
Oct 4 09:52:29 10.0.0.20 [174345.332903] last sysfs file:
/sys/module/configfs/initstate
Oct 4 09:52:29 10.0.0.20 [174345.332952] Modules linked in:
Oct 4 09:52:29 10.0.0.20 netconsole
Oct 4 09:52:29 10.0.0.20 configfs
Oct 4 09:52:29 10.0.0.20 xt_tcpudp
Oct 4 09:52:29 10.0.0.20 nf_conntrack_ipv4
Oct 4 09:52:29 10.0.0.20 nf_defrag_ipv4
Oct 4 09:52:29 10.0.0.20 xt_state
Oct 4 09:52:29 10.0.0.20 nf_conntrack
Oct 4 09:52:29 10.0.0.20 xt_physdev
Oct 4 09:52:29 10.0.0.20 iptable_filter
Oct 4 09:52:29 10.0.0.20 ip_tables
Oct 4 09:52:29 10.0.0.20 x_tables
Oct 4 09:52:29 10.0.0.20 bridge
Oct 4 09:52:29 10.0.0.20 stp
Oct 4 09:52:29 10.0.0.20 xen_evtchn
Oct 4 09:52:29 10.0.0.20 xenfs
Oct 4 09:52:29 10.0.0.20 nfsd
Oct 4 09:52:29 10.0.0.20 exportfs
Oct 4 09:52:29 10.0.0.20 nfs
Oct 4 09:52:29 10.0.0.20 lockd
Oct 4 09:52:29 10.0.0.20 fscache
Oct 4 09:52:29 10.0.0.20 nfs_acl
Oct 4 09:52:29 10.0.0.20 auth_rpcgss
Oct 4 09:52:29 10.0.0.20 sunrpc
Oct 4 09:52:29 10.0.0.20 loop
Oct 4 09:52:29 10.0.0.20 snd_pcm
Oct 4 09:52:29 10.0.0.20 snd_timer
Oct 4 09:52:29 10.0.0.20 snd
Oct 4 09:52:29 10.0.0.20 soundcore
Oct 4 09:52:29 10.0.0.20 psmouse
Oct 4 09:52:29 10.0.0.20 pl2303
Oct 4 09:52:29 10.0.0.20 snd_page_alloc
Oct 4 09:52:29 10.0.0.20 usbserial
Oct 4 09:52:29 10.0.0.20 serio_raw
Oct 4 09:52:29 10.0.0.20 pcspkr
Oct 4 09:52:29 10.0.0.20 shpchp
Oct 4 09:52:29 10.0.0.20 evdev
Oct 4 09:52:29 10.0.0.20 dcdbas
Oct 4 09:52:29 10.0.0.20 pci_hotplug
Oct 4 09:52:29 10.0.0.20 i2c_piix4
Oct 4 09:52:29 10.0.0.20 i2c_core
Oct 4 09:52:29 10.0.0.20 processor
Oct 4 09:52:29 10.0.0.20 acpi_processor
Oct 4 09:52:29 10.0.0.20 button
Oct 4 09:52:29 10.0.0.20 ext3
Oct 4 09:52:29 10.0.0.20 jbd
Oct 4 09:52:29 10.0.0.20 mbcache
Oct 4 09:52:29 10.0.0.20 sg
Oct 4 09:52:29 10.0.0.20 sr_mod
Oct 4 09:52:29 10.0.0.20 cdrom
Oct 4 09:52:29 10.0.0.20 sd_mod
Oct 4 09:52:29 10.0.0.20 crc_t10dif
Oct 4 09:52:29 10.0.0.20 ata_generic
Oct 4 09:52:29 10.0.0.20 ohci_hcd
Oct 4 09:52:29 10.0.0.20 pata_serverworks
Oct 4 09:52:29 10.0.0.20 megaraid_mbox
Oct 4 09:52:29 10.0.0.20 megaraid_mm
Oct 4 09:52:29 10.0.0.20 tg3
Oct 4 09:52:29 10.0.0.20 floppy
Oct 4 09:52:29 10.0.0.20 aacraid
Oct 4 09:52:29 10.0.0.20 libphy
Oct 4 09:52:29 10.0.0.20 qla1280
Oct 4 09:52:29 10.0.0.20 ehci_hcd
Oct 4 09:52:29 10.0.0.20 libata
Oct 4 09:52:29 10.0.0.20 usbcore
Oct 4 09:52:29 10.0.0.20 nls_base
Oct 4 09:52:29 10.0.0.20 thermal
Oct 4 09:52:29 10.0.0.20 scsi_mod
Oct 4 09:52:29 10.0.0.20 thermal_sys
Oct 4 09:52:29 10.0.0.20 [last unloaded: scsi_wait_scan]
Oct 4 09:52:29 10.0.0.20
Oct 4 09:52:29 10.0.0.20 [174345.334427]
Oct 4 09:52:29 10.0.0.20 [174345.334472] Pid: 0, comm: swapper Not
tainted (2.6.32-5-xen-686 #1) PowerEdge 2650
Oct 4 09:52:29 10.0.0.20 [174345.334549] EIP: 0061:[<c119803b>] EFLAGS:
00010282 CPU: 1
Oct 4 09:52:29 10.0.0.20 [174345.334622] EIP is at
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:52:29 10.0.0.20 [174345.334674] EAX: ffffffea EBX: ec847d40
ECX: 00000001 EDX: 00000000
Oct 4 09:52:29 10.0.0.20 [174345.334725] ESI: 00007ff0 EDI: e9976000
EBP: 18375001 ESP: ec847d10
Oct 4 09:52:29 10.0.0.20 [174345.334777] DS: 007b ES: 007b FS: 00d8
GS: 00e0 SS: 0069
Oct 4 09:52:29 10.0.0.20 [174345.334828] Process swapper (pid: 0,
ti=ec846000 task=ec838880 task.ti=ec846000)
Oct 4 09:52:29 10.0.0.20 [174345.334900] Stack:
Oct 4 09:52:29 10.0.0.20 [174345.334953] 000020d1
Oct 4 09:52:29 10.0.0.20 c2424798
Oct 4 09:52:29 10.0.0.20 000000ba
Oct 4 09:52:29 10.0.0.20 c2eaea20
Oct 4 09:52:29 10.0.0.20 c339fea0
Oct 4 09:52:29 10.0.0.20 c20d1000
Oct 4 09:52:29 10.0.0.20 c20d1000
Oct 4 09:52:29 10.0.0.20 00000000
Oct 4 09:52:29 10.0.0.20
Oct 4 09:52:29 10.0.0.20 [174345.335118] <0>
Oct 4 09:52:29 10.0.0.20 e9975000
Oct 4 09:52:29 10.0.0.20 00000000
Oct 4 09:52:29 10.0.0.20 000000ba
Oct 4 09:52:29 10.0.0.20 117f0000
Oct 4 09:52:29 10.0.0.20 18375001
Oct 4 09:52:29 10.0.0.20 00000000
Oct 4 09:52:29 10.0.0.20 000020d1
Oct 4 09:52:29 10.0.0.20 00000000
Oct 4 09:52:29 10.0.0.20
Oct 4 09:52:29 10.0.0.20 [174345.335327] <0>
Oct 4 09:52:29 10.0.0.20 ed811c70
Oct 4 09:52:29 10.0.0.20 ed811da8
Oct 4 09:52:29 10.0.0.20 ed81161c
Oct 4 09:52:29 10.0.0.20 0006a875
Oct 4 09:52:29 10.0.0.20 c11a49ca
Oct 4 09:52:29 10.0.0.20 117fe518
Oct 4 09:52:29 10.0.0.20 00009e8f
Oct 4 09:52:29 10.0.0.20 117fe518
Oct 4 09:52:29 10.0.0.20
Oct 4 09:52:29 10.0.0.20 [174345.335573] Call Trace:
Oct 4 09:52:29 10.0.0.20 [174345.335625] [<c11a49ca>] ?
net_tx_action+0x1c8/0xeb1
Oct 4 09:52:29 10.0.0.20 [174345.335682] [<c10211e3>] ?
pvclock_clocksource_read+0xf9/0x10f
Oct 4 09:52:29 10.0.0.20 [174345.335740] [<c113d59e>] ?
cpumask_next_and+0x23/0x33
Oct 4 09:52:29 10.0.0.20 [174345.335795] [<c102f8c8>] ?
find_busiest_group+0x2e9/0x6e6
Oct 4 09:52:29 10.0.0.20 [174345.335852] [<c1034131>] ?
rebalance_domains+0x149/0x3e1
Oct 4 09:52:29 10.0.0.20 [174345.335908] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:52:29 10.0.0.20 [174345.335963] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:52:29 10.0.0.20 [174345.336031] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:52:29 10.0.0.20 [174345.336110] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:52:29 10.0.0.20 [174345.336176] [<c10066d7>] ?
xen_restore_fl_direct_end+0x0/0x1
Oct 4 09:52:29 10.0.0.20 [174345.336247] [<c128dca1>] ?
_spin_unlock_irqrestore+0xd/0xf
Oct 4 09:52:29 10.0.0.20 [174345.336318] [<c103b17e>] ?
tasklet_action+0x67/0xad
Oct 4 09:52:29 10.0.0.20 [174345.336384] [<c103c2e0>] ?
__do_softirq+0xaa/0x151
Oct 4 09:52:29 10.0.0.20 [174345.336437] [<c103c3b8>] ?
do_softirq+0x31/0x3c
Oct 4 09:52:29 10.0.0.20 [174345.336490] [<c103c48e>] ?
irq_exit+0x26/0x58
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1198c15>] ?
xen_evtchn_do_upcall+0x22/0x2c
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1009abf>] ?
xen_do_upcall+0x7/0xc
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10023a7>] ?
hypercall_page+0x3a7/0x1001
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1006039>] ?
xen_safe_halt+0xf/0x1b
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1004137>] ?
xen_idle+0x23/0x30
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10080a8>] ?
cpu_idle+0x89/0xa5
Oct 4 09:52:29 10.0.0.20 [174345.336513] Code:
Oct 4 09:52:29 10.0.0.20 00
Oct 4 09:52:29 10.0.0.20 31
Oct 4 09:52:29 10.0.0.20 d2
Oct 4 09:52:29 10.0.0.20 8b
Oct 4 09:52:29 10.0.0.20 04
Oct 4 09:52:29 10.0.0.20 24
Oct 4 09:52:29 10.0.0.20 89
Oct 4 09:52:29 10.0.0.20 6c
Oct 4 09:52:29 10.0.0.20 24
Oct 4 09:52:29 10.0.0.20 30
Oct 4 09:52:29 10.0.0.20 8d
Oct 4 09:52:29 10.0.0.20 5c
Oct 4 09:52:29 10.0.0.20 24
Oct 4 09:52:29 10.0.0.20 30
Oct 4 09:52:29 10.0.0.20 c7
Oct 4 09:52:29 10.0.0.20 44
Oct 4 09:52:29 10.0.0.20 24
Oct 4 09:52:29 10.0.0.20 34
Oct 4 09:52:29 10.0.0.20 00
Oct 4 09:52:29 10.0.0.20 last message repeated 3 times
Oct 4 09:52:29 10.0.0.20 89
Oct 4 09:52:29 10.0.0.20 44
Oct 4 09:52:29 10.0.0.20 24
Oct 4 09:52:29 10.0.0.20 38
Oct 4 09:52:29 10.0.0.20 c7
Oct 4 09:52:29 10.0.0.20 44
Oct 4 09:52:29 10.0.0.20 24
Oct 4 09:52:29 10.0.0.20 3c
Oct 4 09:52:29 10.0.0.20 00
Oct 4 09:52:29 10.0.0.20 last message repeated 3 times
Oct 4 09:52:29 10.0.0.20 e8
Oct 4 09:52:29 10.0.0.20 e9
Oct 4 09:52:29 10.0.0.20 9f
Oct 4 09:52:29 10.0.0.20 e6
Oct 4 09:52:29 10.0.0.20 ff
Oct 4 09:52:29 10.0.0.20 85
Oct 4 09:52:29 10.0.0.20 c0
Oct 4 09:52:29 10.0.0.20 74
Oct 4 09:52:29 10.0.0.20 04
Oct 4 09:52:29 10.0.0.20 0b
Oct 4 09:52:29 10.0.0.20 eb
Oct 4 09:52:29 10.0.0.20 fe
Oct 4 09:52:29 10.0.0.20 8b
Oct 4 09:52:29 10.0.0.20 54
Oct 4 09:52:29 10.0.0.20 24
Oct 4 09:52:29 10.0.0.20 0c
Oct 4 09:52:29 10.0.0.20 8b
Oct 4 09:52:29 10.0.0.20 42
Oct 4 09:52:29 10.0.0.20 10
Oct 4 09:52:29 10.0.0.20 8b
Oct 4 09:52:29 10.0.0.20 54
Oct 4 09:52:29 10.0.0.20 24
Oct 4 09:52:29 10.0.0.20 10
Oct 4 09:52:29 10.0.0.20 89
Oct 4 09:52:29 10.0.0.20 42
Oct 4 09:52:29 10.0.0.20 10
Oct 4 09:52:29 10.0.0.20 0f
Oct 4 09:52:29 10.0.0.20 ba
Oct 4 09:52:29 10.0.0.20 2a
Oct 4 09:52:29 10.0.0.20
Oct 4 09:52:29 10.0.0.20 [174345.336513] EIP: [<c119803b>]
Oct 4 09:52:29 10.0.0.20 gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:52:29 10.0.0.20 SS:ESP 0069:ec847d10
Oct 4 09:52:29 10.0.0.20 [174345.336513] ---[ end trace
12e6e134d9f4fe6d ]---
Oct 4 09:52:29 10.0.0.20 [174345.336513] Kernel panic - not syncing:
Fatal exception in interrupt
Oct 4 09:52:29 10.0.0.20 [174345.336513] Pid: 0, comm: swapper Tainted:
G D 2.6.32-5-xen-686 #1
Oct 4 09:52:29 10.0.0.20 [174345.336513] Call Trace:
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c128c191>] ? panic+0x38/0xe4
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c128e799>] ?
oops_end+0x91/0x9d
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1009f21>] ?
do_invalid_op+0x0/0x75
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1009f8d>] ?
do_invalid_op+0x6c/0x75
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1092fc1>] ?
__alloc_pages_nodemask+0xf3/0x4e0
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c128de8b>] ?
error_code+0x73/0x78
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c11a49ca>] ?
net_tx_action+0x1c8/0xeb1
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10211e3>] ?
pvclock_clocksource_read+0xf9/0x10f
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c113d59e>] ?
cpumask_next_and+0x23/0x33
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c102f8c8>] ?
find_busiest_group+0x2e9/0x6e6
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1034131>] ?
rebalance_domains+0x149/0x3e1
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10066d7>] ?
xen_restore_fl_direct_end+0x0/0x1
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c128dca1>] ?
_spin_unlock_irqrestore+0xd/0xf
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c103b17e>] ?
tasklet_action+0x67/0xad
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c103c2e0>] ?
__do_softirq+0xaa/0x151
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c103c3b8>] ?
do_softirq+0x31/0x3c
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c103c48e>] ?
irq_exit+0x26/0x58
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1198c15>] ?
xen_evtchn_do_upcall+0x22/0x2c
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1009abf>] ?
xen_do_upcall+0x7/0xc
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10023a7>] ?
hypercall_page+0x3a7/0x1001
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1006039>] ?
xen_safe_halt+0xf/0x1b
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c1004137>] ?
xen_idle+0x23/0x30
Oct 4 09:52:29 10.0.0.20 [174345.336513] [<c10080a8>] ?
cpu_idle+0x89/0xa5
Oct 4 09:53:33 10.0.0.20 [174410.032507] BUG: soft lockup - CPU#1 stuck
for 61s! [swapper:0]
Oct 4 09:53:33 10.0.0.20 [174410.032507] Modules linked in:
Oct 4 09:53:33 10.0.0.20 netconsole
Oct 4 09:53:33 10.0.0.20 configfs
Oct 4 09:53:33 10.0.0.20 xt_tcpudp
Oct 4 09:53:33 10.0.0.20 nf_conntrack_ipv4
Oct 4 09:53:33 10.0.0.20 nf_defrag_ipv4
Oct 4 09:53:33 10.0.0.20 xt_state
Oct 4 09:53:33 10.0.0.20 nf_conntrack
Oct 4 09:53:33 10.0.0.20 xt_physdev
Oct 4 09:53:33 10.0.0.20 iptable_filter
Oct 4 09:53:33 10.0.0.20 ip_tables
Oct 4 09:53:33 10.0.0.20 x_tables
Oct 4 09:53:33 10.0.0.20 bridge
Oct 4 09:53:33 10.0.0.20 stp
Oct 4 09:53:33 10.0.0.20 xen_evtchn
Oct 4 09:53:33 10.0.0.20 xenfs
Oct 4 09:53:33 10.0.0.20 nfsd
Oct 4 09:53:33 10.0.0.20 exportfs
Oct 4 09:53:33 10.0.0.20 nfs
Oct 4 09:53:33 10.0.0.20 lockd
Oct 4 09:53:33 10.0.0.20 fscache
Oct 4 09:53:33 10.0.0.20 nfs_acl
Oct 4 09:53:33 10.0.0.20 auth_rpcgss
Oct 4 09:53:33 10.0.0.20 sunrpc
Oct 4 09:53:33 10.0.0.20 loop
Oct 4 09:53:33 10.0.0.20 snd_pcm
Oct 4 09:53:33 10.0.0.20 snd_timer
Oct 4 09:53:33 10.0.0.20 snd
Oct 4 09:53:33 10.0.0.20 soundcore
Oct 4 09:53:33 10.0.0.20 psmouse
Oct 4 09:53:33 10.0.0.20 pl2303
Oct 4 09:53:33 10.0.0.20 snd_page_alloc
Oct 4 09:53:33 10.0.0.20 usbserial
Oct 4 09:53:33 10.0.0.20 serio_raw
Oct 4 09:53:33 10.0.0.20 pcspkr
Oct 4 09:53:33 10.0.0.20 shpchp
Oct 4 09:53:33 10.0.0.20 evdev
Oct 4 09:53:33 10.0.0.20 dcdbas
Oct 4 09:53:33 10.0.0.20 pci_hotplug
Oct 4 09:53:33 10.0.0.20 i2c_piix4
Oct 4 09:53:33 10.0.0.20 i2c_core
Oct 4 09:53:33 10.0.0.20 processor
Oct 4 09:53:33 10.0.0.20 acpi_processor
Oct 4 09:53:33 10.0.0.20 button
Oct 4 09:53:33 10.0.0.20 ext3
Oct 4 09:53:33 10.0.0.20 jbd
Oct 4 09:53:33 10.0.0.20 mbcache
Oct 4 09:53:33 10.0.0.20 sg
Oct 4 09:53:33 10.0.0.20 sr_mod
Oct 4 09:53:33 10.0.0.20 cdrom
Oct 4 09:53:33 10.0.0.20 sd_mod
Oct 4 09:53:33 10.0.0.20 crc_t10dif
Oct 4 09:53:33 10.0.0.20 ata_generic
Oct 4 09:53:33 10.0.0.20 ohci_hcd
Oct 4 09:53:33 10.0.0.20 pata_serverworks
Oct 4 09:53:33 10.0.0.20 megaraid_mbox
Oct 4 09:53:33 10.0.0.20 megaraid_mm
Oct 4 09:53:33 10.0.0.20 tg3
Oct 4 09:53:33 10.0.0.20 floppy
Oct 4 09:53:33 10.0.0.20 aacraid
Oct 4 09:53:33 10.0.0.20 libphy
Oct 4 09:53:33 10.0.0.20 qla1280
Oct 4 09:53:33 10.0.0.20 ehci_hcd
Oct 4 09:53:33 10.0.0.20 libata
Oct 4 09:53:33 10.0.0.20 usbcore
Oct 4 09:53:33 10.0.0.20 nls_base
Oct 4 09:53:33 10.0.0.20 thermal
Oct 4 09:53:33 10.0.0.20 scsi_mod
Oct 4 09:53:33 10.0.0.20 thermal_sys
Oct 4 09:53:33 10.0.0.20 [last unloaded: scsi_wait_scan]
Oct 4 09:53:33 10.0.0.20
Oct 4 09:53:33 10.0.0.20 [174410.032507]
Oct 4 09:53:33 10.0.0.20 [174410.032507] Pid: 0, comm: swapper Tainted:
G D (2.6.32-5-xen-686 #1) PowerEdge 2650
Oct 4 09:53:33 10.0.0.20 [174410.032507] EIP: 0061:[<c1059e72>] EFLAGS:
00000202 CPU: 1
Oct 4 09:53:33 10.0.0.20 [174410.032507] EIP is at
smp_call_function_many+0xed/0x17c
Oct 4 09:53:33 10.0.0.20 [174410.032507] EAX: c145a6dc EBX: 00000002
ECX: 00000003 EDX: 03cd9000
Oct 4 09:53:33 10.0.0.20 [174410.032507] ESI: c51336dc EDI: 00000001
EBP: c13fa2b4 ESP: ec847ba0
Oct 4 09:53:33 10.0.0.20 [174410.032507] DS: 007b ES: 007b FS: 00d8
GS: 00e0 SS: 0069
Oct 4 09:53:33 10.0.0.20 [174410.032507] CR0: 8005003b CR2: 094c7808
CR3: 0a1c5000 CR4: 00000660
Oct 4 09:53:33 10.0.0.20 [174410.032507] DR0: 00000000 DR1: 00000000
DR2: 00000000 DR3: 00000000
Oct 4 09:53:33 10.0.0.20 [174410.032507] DR6: ffff0ff0 DR7: 00000400
Oct 4 09:53:33 10.0.0.20 [174410.032507] Call Trace:
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1006d5c>] ?
stop_self+0x0/0x30
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1059f1a>] ?
smp_call_function+0x19/0x1f
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c10036be>] ?
xen_reboot+0x14/0x28
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1003705>] ?
xen_panic_event+0xa/0xd
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c128f990>] ?
notifier_call_chain+0x2a/0x47
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c128f9c0>] ?
atomic_notifier_call_chain+0xc/0xf
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c128c1af>] ? panic+0x56/0xe4
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c128e799>] ?
oops_end+0x91/0x9d
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1009f21>] ?
do_invalid_op+0x0/0x75
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1009f8d>] ?
do_invalid_op+0x6c/0x75
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1092fc1>] ?
__alloc_pages_nodemask+0xf3/0x4e0
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c128de8b>] ?
error_code+0x73/0x78
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c11a49ca>] ?
net_tx_action+0x1c8/0xeb1
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c10211e3>] ?
pvclock_clocksource_read+0xf9/0x10f
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c113d59e>] ?
cpumask_next_and+0x23/0x33
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c102f8c8>] ?
find_busiest_group+0x2e9/0x6e6
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1034131>] ?
rebalance_domains+0x149/0x3e1
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c10066d7>] ?
xen_restore_fl_direct_end+0x0/0x1
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c128dca1>] ?
_spin_unlock_irqrestore+0xd/0xf
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c103b17e>] ?
tasklet_action+0x67/0xad
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c103c2e0>] ?
__do_softirq+0xaa/0x151
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c103c3b8>] ?
do_softirq+0x31/0x3c
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c103c48e>] ?
irq_exit+0x26/0x58
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1198c15>] ?
xen_evtchn_do_upcall+0x22/0x2c
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1009abf>] ?
xen_do_upcall+0x7/0xc
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c10023a7>] ?
hypercall_page+0x3a7/0x1001
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1006039>] ?
xen_safe_halt+0xf/0x1b
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c1004137>] ?
xen_idle+0x23/0x30
Oct 4 09:53:33 10.0.0.20 [174410.032507] [<c10080a8>] ?
cpu_idle+0x89/0xa5
Oct 4 09:54:39 10.0.0.20 [174475.532508] BUG: soft lockup - CPU#1 stuck
for 61s! [swapper:0]
Oct 4 09:54:39 10.0.0.20 [174475.532508] Modules linked in:
Oct 4 09:54:39 10.0.0.20 netconsole
Oct 4 09:54:39 10.0.0.20 configfs
Oct 4 09:54:39 10.0.0.20 xt_tcpudp
Oct 4 09:54:39 10.0.0.20 nf_conntrack_ipv4
Oct 4 09:54:39 10.0.0.20 nf_defrag_ipv4
Oct 4 09:54:39 10.0.0.20 xt_state
Oct 4 09:54:39 10.0.0.20 nf_conntrack
Oct 4 09:54:39 10.0.0.20 xt_physdev
Oct 4 09:54:39 10.0.0.20 iptable_filter
Oct 4 09:54:39 10.0.0.20 ip_tables
Oct 4 09:54:39 10.0.0.20 x_tables
Oct 4 09:54:39 10.0.0.20 bridge
Oct 4 09:54:39 10.0.0.20 stp
Oct 4 09:54:39 10.0.0.20 xen_evtchn
Oct 4 09:54:39 10.0.0.20 xenfs
Oct 4 09:54:39 10.0.0.20 nfsd
Oct 4 09:54:39 10.0.0.20 exportfs
Oct 4 09:54:39 10.0.0.20 nfs
Oct 4 09:54:39 10.0.0.20 lockd
Oct 4 09:54:39 10.0.0.20 fscache
Oct 4 09:54:39 10.0.0.20 nfs_acl
Oct 4 09:54:39 10.0.0.20 auth_rpcgss
Oct 4 09:54:39 10.0.0.20 sunrpc
Oct 4 09:54:39 10.0.0.20 loop
Oct 4 09:54:39 10.0.0.20 snd_pcm
Oct 4 09:54:39 10.0.0.20 snd_timer
Oct 4 09:54:39 10.0.0.20 snd
Oct 4 09:54:39 10.0.0.20 soundcore
Oct 4 09:54:39 10.0.0.20 psmouse
Oct 4 09:54:39 10.0.0.20 pl2303
Oct 4 09:54:39 10.0.0.20 snd_page_alloc
Oct 4 09:54:39 10.0.0.20 usbserial
Oct 4 09:54:39 10.0.0.20 serio_raw
Oct 4 09:54:39 10.0.0.20 pcspkr
Oct 4 09:54:39 10.0.0.20 shpchp
Oct 4 09:54:39 10.0.0.20 evdev
Oct 4 09:54:39 10.0.0.20 dcdbas
Oct 4 09:54:39 10.0.0.20 pci_hotplug
Oct 4 09:54:39 10.0.0.20 i2c_piix4
Oct 4 09:54:39 10.0.0.20 i2c_core
Oct 4 09:54:39 10.0.0.20 processor
Oct 4 09:54:39 10.0.0.20 acpi_processor
Oct 4 09:54:39 10.0.0.20 button
Oct 4 09:54:39 10.0.0.20 ext3
Oct 4 09:54:39 10.0.0.20 jbd
Oct 4 09:54:39 10.0.0.20 mbcache
Oct 4 09:54:39 10.0.0.20 sg
Oct 4 09:54:39 10.0.0.20 sr_mod
Oct 4 09:54:39 10.0.0.20 cdrom
Oct 4 09:54:39 10.0.0.20 sd_mod
Oct 4 09:54:39 10.0.0.20 crc_t10dif
Oct 4 09:54:39 10.0.0.20 ata_generic
Oct 4 09:54:39 10.0.0.20 ohci_hcd
Oct 4 09:54:39 10.0.0.20 pata_serverworks
Oct 4 09:54:39 10.0.0.20 megaraid_mbox
Oct 4 09:54:39 10.0.0.20 megaraid_mm
Oct 4 09:54:39 10.0.0.20 tg3
Oct 4 09:54:39 10.0.0.20 floppy
Oct 4 09:54:39 10.0.0.20 aacraid
Oct 4 09:54:39 10.0.0.20 libphy
Oct 4 09:54:39 10.0.0.20 qla1280
Oct 4 09:54:39 10.0.0.20 ehci_hcd
Oct 4 09:54:39 10.0.0.20 libata
Oct 4 09:54:39 10.0.0.20 usbcore
Oct 4 09:54:39 10.0.0.20 nls_base
Oct 4 09:54:39 10.0.0.20 thermal
Oct 4 09:54:39 10.0.0.20 scsi_mod
Oct 4 09:54:39 10.0.0.20 thermal_sys
Oct 4 09:54:39 10.0.0.20 [last unloaded: scsi_wait_scan]
Oct 4 09:54:39 10.0.0.20
Oct 4 09:54:39 10.0.0.20 [174475.532508]
Oct 4 09:54:39 10.0.0.20 [174475.532508] Pid: 0, comm: swapper Tainted:
G D (2.6.32-5-xen-686 #1) PowerEdge 2650
Oct 4 09:54:39 10.0.0.20 [174475.532508] EIP: 0061:[<c1059e6e>] EFLAGS:
00000202 CPU: 1
Oct 4 09:54:39 10.0.0.20 [174475.532508] EIP is at
smp_call_function_many+0xe9/0x17c
Oct 4 09:54:39 10.0.0.20 [174475.532508] EAX: c145a6dc EBX: 00000002
ECX: 00000003 EDX: 03cd9000
Oct 4 09:54:39 10.0.0.20 [174475.532508] ESI: c51336dc EDI: 00000001
EBP: c13fa2b4 ESP: ec847ba0
Oct 4 09:54:39 10.0.0.20 [174475.532508] DS: 007b ES: 007b FS: 00d8
GS: 00e0 SS: 0069
Oct 4 09:54:39 10.0.0.20 [174475.532508] CR0: 8005003b CR2: 094c7808
CR3: 0a1c5000 CR4: 00000660
Oct 4 09:54:39 10.0.0.20 [174475.532508] DR0: 00000000 DR1: 00000000
DR2: 00000000 DR3: 00000000
Oct 4 09:54:39 10.0.0.20 [174475.532508] DR6: ffff0ff0 DR7: 00000400
Oct 4 09:54:39 10.0.0.20 [174475.532508] Call Trace:
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1006d5c>] ?
stop_self+0x0/0x30
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1059f1a>] ?
smp_call_function+0x19/0x1f
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c10036be>] ?
xen_reboot+0x14/0x28
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1003705>] ?
xen_panic_event+0xa/0xd
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c128f990>] ?
notifier_call_chain+0x2a/0x47
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c128f9c0>] ?
atomic_notifier_call_chain+0xc/0xf
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c128c1af>] ? panic+0x56/0xe4
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c128e799>] ?
oops_end+0x91/0x9d
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1009f21>] ?
do_invalid_op+0x0/0x75
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1009f8d>] ?
do_invalid_op+0x6c/0x75
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1092fc1>] ?
__alloc_pages_nodemask+0xf3/0x4e0
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c128de8b>] ?
error_code+0x73/0x78
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c11a49ca>] ?
net_tx_action+0x1c8/0xeb1
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c10211e3>] ?
pvclock_clocksource_read+0xf9/0x10f
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c113d59e>] ?
cpumask_next_and+0x23/0x33
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c102f8c8>] ?
find_busiest_group+0x2e9/0x6e6
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1034131>] ?
rebalance_domains+0x149/0x3e1
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c10066d7>] ?
xen_restore_fl_direct_end+0x0/0x1
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c128dca1>] ?
_spin_unlock_irqrestore+0xd/0xf
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c103b17e>] ?
tasklet_action+0x67/0xad
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c103c2e0>] ?
__do_softirq+0xaa/0x151
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c103c3b8>] ?
do_softirq+0x31/0x3c
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c103c48e>] ?
irq_exit+0x26/0x58
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1198c15>] ?
xen_evtchn_do_upcall+0x22/0x2c
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1009abf>] ?
xen_do_upcall+0x7/0xc
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c10023a7>] ?
hypercall_page+0x3a7/0x1001
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1006039>] ?
xen_safe_halt+0xf/0x1b
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c1004137>] ?
xen_idle+0x23/0x30
Oct 4 09:54:39 10.0.0.20 [174475.532508] [<c10080a8>] ?
cpu_idle+0x89/0xa5
Oct 4 09:55:44 10.0.0.20 [174541.028507] BUG: soft lockup - CPU#1 stuck
for 61s! [swapper:0]
Oct 4 09:55:44 10.0.0.20 [174541.028507] Modules linked in:
Oct 4 09:55:44 10.0.0.20 netconsole
Oct 4 09:55:44 10.0.0.20 configfs
Oct 4 09:55:44 10.0.0.20 xt_tcpudp
Oct 4 09:55:44 10.0.0.20 nf_conntrack_ipv4
Oct 4 09:55:44 10.0.0.20 nf_defrag_ipv4
Oct 4 09:55:44 10.0.0.20 xt_state
Oct 4 09:55:44 10.0.0.20 nf_conntrack
Oct 4 09:55:44 10.0.0.20 xt_physdev
Oct 4 09:55:44 10.0.0.20 iptable_filter
Oct 4 09:55:44 10.0.0.20 ip_tables
Oct 4 09:55:44 10.0.0.20 x_tables
Oct 4 09:55:44 10.0.0.20 bridge
Oct 4 09:55:44 10.0.0.20 stp
Oct 4 09:55:44 10.0.0.20 xen_evtchn
Oct 4 09:55:44 10.0.0.20 xenfs
Oct 4 09:55:44 10.0.0.20 nfsd
Oct 4 09:55:44 10.0.0.20 exportfs
Oct 4 09:55:44 10.0.0.20 nfs
Oct 4 09:55:44 10.0.0.20 lockd
Oct 4 09:55:44 10.0.0.20 fscache
Oct 4 09:55:44 10.0.0.20 nfs_acl
Oct 4 09:55:44 10.0.0.20 auth_rpcgss
Oct 4 09:55:44 10.0.0.20 sunrpc
Oct 4 09:55:44 10.0.0.20 loop
Oct 4 09:55:44 10.0.0.20 snd_pcm
Oct 4 09:55:44 10.0.0.20 snd_timer
Oct 4 09:55:44 10.0.0.20 snd
Oct 4 09:55:44 10.0.0.20 soundcore
Oct 4 09:55:44 10.0.0.20 psmouse
Oct 4 09:55:44 10.0.0.20 pl2303
Oct 4 09:55:44 10.0.0.20 snd_page_alloc
Oct 4 09:55:44 10.0.0.20 usbserial
Oct 4 09:55:44 10.0.0.20 serio_raw
Oct 4 09:55:44 10.0.0.20 pcspkr
Oct 4 09:55:44 10.0.0.20 shpchp
Oct 4 09:55:44 10.0.0.20 evdev
Oct 4 09:55:44 10.0.0.20 dcdbas
Oct 4 09:55:44 10.0.0.20 pci_hotplug
Oct 4 09:55:44 10.0.0.20 i2c_piix4
Oct 4 09:55:44 10.0.0.20 i2c_core
Oct 4 09:55:44 10.0.0.20 processor
Oct 4 09:55:44 10.0.0.20 acpi_processor
Oct 4 09:55:44 10.0.0.20 button
Oct 4 09:55:44 10.0.0.20 ext3
Oct 4 09:55:44 10.0.0.20 jbd
Oct 4 09:55:44 10.0.0.20 mbcache
Oct 4 09:55:44 10.0.0.20 sg
Oct 4 09:55:44 10.0.0.20 sr_mod
Oct 4 09:55:44 10.0.0.20 cdrom
Oct 4 09:55:44 10.0.0.20 sd_mod
Oct 4 09:55:44 10.0.0.20 crc_t10dif
Oct 4 09:55:44 10.0.0.20 ata_generic
Oct 4 09:55:44 10.0.0.20 ohci_hcd
Oct 4 09:55:44 10.0.0.20 pata_serverworks
Oct 4 09:55:44 10.0.0.20 megaraid_mbox
Oct 4 09:55:44 10.0.0.20 megaraid_mm
Oct 4 09:55:44 10.0.0.20 tg3
Oct 4 09:55:44 10.0.0.20 floppy
Oct 4 09:55:44 10.0.0.20 aacraid
Oct 4 09:55:44 10.0.0.20 libphy
Oct 4 09:55:44 10.0.0.20 qla1280
Oct 4 09:55:44 10.0.0.20 ehci_hcd
Oct 4 09:55:44 10.0.0.20 libata
Oct 4 09:55:44 10.0.0.20 usbcore
Oct 4 09:55:44 10.0.0.20 nls_base
Oct 4 09:55:44 10.0.0.20 thermal
Oct 4 09:55:44 10.0.0.20 scsi_mod
Oct 4 09:55:44 10.0.0.20 thermal_sys
Oct 4 09:55:44 10.0.0.20 [last unloaded: scsi_wait_scan]
Oct 4 09:55:44 10.0.0.20
Oct 4 09:55:44 10.0.0.20 [174541.028507]
Oct 4 09:55:44 10.0.0.20 [174541.028507] Pid: 0, comm: swapper Tainted:
G D (2.6.32-5-xen-686 #1) PowerEdge 2650
Oct 4 09:55:44 10.0.0.20 [174541.028507] EIP: 0061:[<c1059e6e>] EFLAGS:
00000202 CPU: 1
Oct 4 09:55:44 10.0.0.20 [174541.028507] EIP is at
smp_call_function_many+0xe9/0x17c
Oct 4 09:55:44 10.0.0.20 [174541.028507] EAX: c145a6dc EBX: 00000002
ECX: 00000003 EDX: 03cd9000
Oct 4 09:55:44 10.0.0.20 [174541.028507] ESI: c51336dc EDI: 00000001
EBP: c13fa2b4 ESP: ec847ba0
Oct 4 09:55:44 10.0.0.20 [174541.028507] DS: 007b ES: 007b FS: 00d8
GS: 00e0 SS: 0069
Oct 4 09:55:44 10.0.0.20 [174541.028507] CR0: 8005003b CR2: 094c7808
CR3: 0a1c5000 CR4: 00000660
Oct 4 09:55:44 10.0.0.20 [174541.028507] DR0: 00000000 DR1: 00000000
DR2: 00000000 DR3: 00000000
Oct 4 09:55:44 10.0.0.20 [174541.028507] DR6: ffff0ff0 DR7: 00000400
Oct 4 09:55:44 10.0.0.20 [174541.028507] Call Trace:
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1006d5c>] ?
stop_self+0x0/0x30
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1059f1a>] ?
smp_call_function+0x19/0x1f
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c10036be>] ?
xen_reboot+0x14/0x28
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1003705>] ?
xen_panic_event+0xa/0xd
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c128f990>] ?
notifier_call_chain+0x2a/0x47
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c128f9c0>] ?
atomic_notifier_call_chain+0xc/0xf
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c128c1af>] ? panic+0x56/0xe4
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c128e799>] ?
oops_end+0x91/0x9d
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1009f21>] ?
do_invalid_op+0x0/0x75
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1009f8d>] ?
do_invalid_op+0x6c/0x75
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1092fc1>] ?
__alloc_pages_nodemask+0xf3/0x4e0
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c128de8b>] ?
error_code+0x73/0x78
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c11a49ca>] ?
net_tx_action+0x1c8/0xeb1
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c10211e3>] ?
pvclock_clocksource_read+0xf9/0x10f
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c113d59e>] ?
cpumask_next_and+0x23/0x33
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c102f8c8>] ?
find_busiest_group+0x2e9/0x6e6
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1034131>] ?
rebalance_domains+0x149/0x3e1
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c10066d7>] ?
xen_restore_fl_direct_end+0x0/0x1
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c128dca1>] ?
_spin_unlock_irqrestore+0xd/0xf
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c103b17e>] ?
tasklet_action+0x67/0xad
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c103c2e0>] ?
__do_softirq+0xaa/0x151
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c103c3b8>] ?
do_softirq+0x31/0x3c
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c103c48e>] ?
irq_exit+0x26/0x58
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1198c15>] ?
xen_evtchn_do_upcall+0x22/0x2c
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1009abf>] ?
xen_do_upcall+0x7/0xc
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c10023a7>] ?
hypercall_page+0x3a7/0x1001
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1006039>] ?
xen_safe_halt+0xf/0x1b
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c1004137>] ?
xen_idle+0x23/0x30
Oct 4 09:55:44 10.0.0.20 [174541.028507] [<c10080a8>] ?
cpu_idle+0x89/0xa5
Oct 4 09:56:50 10.0.0.20 [174606.528507] BUG: soft lockup - CPU#1 stuck
for 61s! [swapper:0]
Oct 4 09:56:50 10.0.0.20 [174606.528507] Modules linked in:
Oct 4 09:56:50 10.0.0.20 netconsole
Oct 4 09:56:50 10.0.0.20 configfs
Oct 4 09:56:50 10.0.0.20 xt_tcpudp
Oct 4 09:56:50 10.0.0.20 nf_conntrack_ipv4
Oct 4 09:56:50 10.0.0.20 nf_defrag_ipv4
Oct 4 09:56:50 10.0.0.20 xt_state
Oct 4 09:56:50 10.0.0.20 nf_conntrack
Oct 4 09:56:50 10.0.0.20 xt_physdev
Oct 4 09:56:50 10.0.0.20 iptable_filter
Oct 4 09:56:50 10.0.0.20 ip_tables
Oct 4 09:56:50 10.0.0.20 x_tables
Oct 4 09:56:50 10.0.0.20 bridge
Oct 4 09:56:50 10.0.0.20 stp
Oct 4 09:56:50 10.0.0.20 xen_evtchn
Oct 4 09:56:50 10.0.0.20 xenfs
Oct 4 09:56:50 10.0.0.20 nfsd
Oct 4 09:56:50 10.0.0.20 exportfs
Oct 4 09:56:50 10.0.0.20 nfs
Oct 4 09:56:50 10.0.0.20 lockd
Oct 4 09:56:50 10.0.0.20 fscache
Oct 4 09:56:50 10.0.0.20 nfs_acl
Oct 4 09:56:50 10.0.0.20 auth_rpcgss
Oct 4 09:56:50 10.0.0.20 sunrpc
Oct 4 09:56:50 10.0.0.20 loop
Oct 4 09:56:50 10.0.0.20 snd_pcm
Oct 4 09:56:50 10.0.0.20 snd_timer
Oct 4 09:56:50 10.0.0.20 snd
Oct 4 09:56:50 10.0.0.20 soundcore
Oct 4 09:56:50 10.0.0.20 psmouse
Oct 4 09:56:50 10.0.0.20 pl2303
Oct 4 09:56:50 10.0.0.20 snd_page_alloc
Oct 4 09:56:50 10.0.0.20 usbserial
Oct 4 09:56:50 10.0.0.20 serio_raw
Oct 4 09:56:50 10.0.0.20 pcspkr
Oct 4 09:56:50 10.0.0.20 shpchp
Oct 4 09:56:50 10.0.0.20 evdev
Oct 4 09:56:50 10.0.0.20 dcdbas
Oct 4 09:56:50 10.0.0.20 pci_hotplug
Oct 4 09:56:50 10.0.0.20 i2c_piix4
Oct 4 09:56:50 10.0.0.20 i2c_core
Oct 4 09:56:50 10.0.0.20 processor
Oct 4 09:56:50 10.0.0.20 acpi_processor
Oct 4 09:56:50 10.0.0.20 button
Oct 4 09:56:50 10.0.0.20 ext3
Oct 4 09:56:50 10.0.0.20 jbd
Oct 4 09:56:50 10.0.0.20 mbcache
Oct 4 09:56:50 10.0.0.20 sg
Oct 4 09:56:50 10.0.0.20 sr_mod
Oct 4 09:56:50 10.0.0.20 cdrom
Oct 4 09:56:50 10.0.0.20 sd_mod
Oct 4 09:56:50 10.0.0.20 crc_t10dif
Oct 4 09:56:50 10.0.0.20 ata_generic
Oct 4 09:56:50 10.0.0.20 ohci_hcd
Oct 4 09:56:50 10.0.0.20 pata_serverworks
Oct 4 09:56:50 10.0.0.20 megaraid_mbox
Oct 4 09:56:50 10.0.0.20 megaraid_mm
Oct 4 09:56:50 10.0.0.20 tg3
Oct 4 09:56:50 10.0.0.20 floppy
Oct 4 09:56:50 10.0.0.20 aacraid
Oct 4 09:56:50 10.0.0.20 libphy
Oct 4 09:56:50 10.0.0.20 qla1280
Oct 4 09:56:50 10.0.0.20 ehci_hcd
Oct 4 09:56:50 10.0.0.20 libata
Oct 4 09:56:50 10.0.0.20 usbcore
Oct 4 09:56:50 10.0.0.20 nls_base
Oct 4 09:56:50 10.0.0.20 thermal
Oct 4 09:56:50 10.0.0.20 scsi_mod
Oct 4 09:56:50 10.0.0.20 thermal_sys
Oct 4 09:56:50 10.0.0.20 [last unloaded: scsi_wait_scan]
Oct 4 09:56:50 10.0.0.20
Oct 4 09:56:50 10.0.0.20 [174606.528507]
Oct 4 09:56:50 10.0.0.20 [174606.528507] Pid: 0, comm: swapper Tainted:
G D (2.6.32-5-xen-686 #1) PowerEdge 2650
Oct 4 09:56:50 10.0.0.20 [174606.528507] EIP: 0061:[<c1059e6e>] EFLAGS:
00000202 CPU: 1
Oct 4 09:56:50 10.0.0.20 [174606.528507] EIP is at
smp_call_function_many+0xe9/0x17c
Oct 4 09:56:50 10.0.0.20 [174606.528507] EAX: c145a6dc EBX: 00000002
ECX: 00000003 EDX: 03cd9000
Oct 4 09:56:50 10.0.0.20 [174606.528507] ESI: c51336dc EDI: 00000001
EBP: c13fa2b4 ESP: ec847ba0
Oct 4 09:56:50 10.0.0.20 [174606.528507] DS: 007b ES: 007b FS: 00d8
GS: 00e0 SS: 0069
Oct 4 09:56:50 10.0.0.20 [174606.528507] CR0: 8005003b CR2: 094c7808
CR3: 0a1c5000 CR4: 00000660
Oct 4 09:56:50 10.0.0.20 [174606.528507] DR0: 00000000 DR1: 00000000
DR2: 00000000 DR3: 00000000
Oct 4 09:56:50 10.0.0.20 [174606.528507] DR6: ffff0ff0 DR7: 00000400
Oct 4 09:56:50 10.0.0.20 [174606.528507] Call Trace:
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1006d5c>] ?
stop_self+0x0/0x30
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1059f1a>] ?
smp_call_function+0x19/0x1f
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c10036be>] ?
xen_reboot+0x14/0x28
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1003705>] ?
xen_panic_event+0xa/0xd
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c128f990>] ?
notifier_call_chain+0x2a/0x47
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c128f9c0>] ?
atomic_notifier_call_chain+0xc/0xf
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c128c1af>] ? panic+0x56/0xe4
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c128e799>] ?
oops_end+0x91/0x9d
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1009f21>] ?
do_invalid_op+0x0/0x75
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1009f8d>] ?
do_invalid_op+0x6c/0x75
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1092fc1>] ?
__alloc_pages_nodemask+0xf3/0x4e0
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c128de8b>] ?
error_code+0x73/0x78
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c119803b>] ?
gnttab_copy_grant_page+0x17d/0x22c
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c11a49ca>] ?
net_tx_action+0x1c8/0xeb1
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c10211e3>] ?
pvclock_clocksource_read+0xf9/0x10f
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c113d59e>] ?
cpumask_next_and+0x23/0x33
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c102f8c8>] ?
find_busiest_group+0x2e9/0x6e6
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1034131>] ?
rebalance_domains+0x149/0x3e1
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1005fb8>] ?
xen_force_evtchn_callback+0xc/0x10
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c10066e0>] ?
check_events+0x8/0xc
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c10066d7>] ?
xen_restore_fl_direct_end+0x0/0x1
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c128dca1>] ?
_spin_unlock_irqrestore+0xd/0xf
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c103b17e>] ?
tasklet_action+0x67/0xad
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c103c2e0>] ?
__do_softirq+0xaa/0x151
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c103c3b8>] ?
do_softirq+0x31/0x3c
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c103c48e>] ?
irq_exit+0x26/0x58
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1198c15>] ?
xen_evtchn_do_upcall+0x22/0x2c
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1009abf>] ?
xen_do_upcall+0x7/0xc
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c10023a7>] ?
hypercall_page+0x3a7/0x1001
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1006039>] ?
xen_safe_halt+0xf/0x1b
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c1004137>] ?
xen_idle+0x23/0x30
Oct 4 09:56:50 10.0.0.20 [174606.528507] [<c10080a8>] ?
cpu_idle+0x89/0xa5
--- End Message ---