Hello,
I installed sid onto a new machine yesterday (Intel ICH chipset).
2.6.32-5-amd64 boots fine, but 2.6.36-rc6-amd64 makes udev settle
timeout when run from the rcS.d/udev script. As a consequence,
a random combination of PCI devices then won't be available: at some
point, the sound card will be missing, at others there won't be
a network card, sometimes the graphics cards go AWOL, etc.
A few seconds before rcS starts, I always get a BUG (see below) in
modprobe, which does not appear with 2.6.32. I suspect it's related,
but I don't know for sure. The referenced PCI device (last sysfs
file) always seems to be one on PCI bus ff:*.* (see below for
lspci). The device alternatives between those, but the rest of the
BUG and the trace always looks the same.
I am finding this hard to debug because I have not been able to
capture the messages sent to the console; syslog does not get them,
bootlog does not get them, dmesg does not see all of them (e.g. not
the timeout from udevadm settle).
During the settle timeout, if I press ctrl-c, I can get to a shell
and inspect the processes. There are a few modprobe processes, some
in D state, and looking at the PCI IDs embedded in the PCI alias
arguments to modprobe, they all seem to be related to PCI and USB
controllers (PCI bus 00:1*.* basically).
The first thing I did was reduce the timeout to facilitate
debugging, and I only needed to edit /etc/rcS.d/S02udev for that
(append --timeout=20 to udevadm settle). Now I didn't have to wait
180 seconds each time.
Then I tried to get netconsole to work to capture the console
messages, which I could not find in any log file post-boot. However,
even though I moved /etc/rcS.d/S09module-init-tools to
S00module-init-tools, and inserted a sleep 30 at the end of the
script, netconsole would only get loaded well after the timeout
passed.
Yet, the timeout length is controlled from /etc/rcS.d/S02udev,
which is supposed to start afterwards. Weird.
I added 'exec >/lib/init/rw/udev.output' to the udev script and now
managed to capture the output:
Waiting for /dev to be fully populated...
udevadm settle - timeout of 20 seconds reached, the event queue contains:
/sys/devices/pci0000:00/0000:00:03.0/0000:02:00.0/0000:03:00.0/0000:04:00.1 (1264)
/sys/devices/pci0000:00/0000:00:03.0/0000:02:00.0/0000:03:02.0/0000:05:00.1 (1268)
/sys/devices/pci0000:00/0000:00:07.0/0000:06:00.1 (1274)
/sys/devices/pci0000:00/0000:00:1b.0 (1321)
done (timeout).
All of those seem related to *audio* devices (the graphics cards
apparently have audio devices too nowadays), and most often they do,
but occasionally I see another device in there, e.g. just now the
SMBus controller.
I am keen on getting this resolved, so I'll gladly do whatever is
necessary to help this problem get resolved. I can even provide
access to the machine, if it would help.
lspci -k:
00:00.0 Host bridge: Intel Corporation 5520/5500/X58 I/O Hub to ESI Port (rev 13)
Subsystem: ASUSTeK Computer Inc. Device 836b
00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13)
00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13)
00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13)
00:14.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management Registers (rev 13)
Kernel driver in use: i7core_edac
00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13)
00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev 13)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
Subsystem: ASUSTeK Computer Inc. Device 82d4
Kernel driver in use: uhci_hcd
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
Subsystem: ASUSTeK Computer Inc. Device 82d4
Kernel driver in use: uhci_hcd
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
Subsystem: ASUSTeK Computer Inc. Device 82d4
Kernel driver in use: uhci_hcd
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
Subsystem: ASUSTeK Computer Inc. Device 82d4
Kernel driver in use: ehci_hcd
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
Subsystem: ASUSTeK Computer Inc. Device 82ea
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.1 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 2
00:1c.2 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 3
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
Subsystem: ASUSTeK Computer Inc. Device 82d4
Kernel driver in use: uhci_hcd
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
Subsystem: ASUSTeK Computer Inc. Device 82d4
Kernel driver in use: uhci_hcd
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
Subsystem: ASUSTeK Computer Inc. Device 82d4
Kernel driver in use: uhci_hcd
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
Subsystem: ASUSTeK Computer Inc. Device 82d4
Kernel driver in use: ehci_hcd
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
Subsystem: ASUSTeK Computer Inc. Device 82d4
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
Subsystem: ASUSTeK Computer Inc. P5Q Deluxe Motherboard
Kernel driver in use: ahci
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
Subsystem: ASUSTeK Computer Inc. Device 82d4
02:00.0 PCI bridge: nVidia Corporation NF200 PCIe 2.0 switch for mainboards (rev a3)
03:00.0 PCI bridge: nVidia Corporation NF200 PCIe 2.0 switch for mainboards (rev a3)
03:02.0 PCI bridge: nVidia Corporation NF200 PCIe 2.0 switch for mainboards (rev a3)
04:00.0 VGA compatible controller: ATI Technologies Inc RV710 [Radeon HD 4350]
Subsystem: Hightech Information System Ltd. Device 2271
Kernel driver in use: radeon
04:00.1 Audio device: ATI Technologies Inc RV710/730
Subsystem: Hightech Information System Ltd. Device aa38
05:00.0 VGA compatible controller: ATI Technologies Inc RV710 [Radeon HD 4350]
Subsystem: Hightech Information System Ltd. Device 2271
Kernel driver in use: radeon
05:00.1 Audio device: ATI Technologies Inc RV710/730
Subsystem: Hightech Information System Ltd. Device aa38
06:00.0 VGA compatible controller: ATI Technologies Inc RV710 [Radeon HD 4350]
Subsystem: Hightech Information System Ltd. Device 2271
Kernel driver in use: radeon
06:00.1 Audio device: ATI Technologies Inc RV710/730
Subsystem: Hightech Information System Ltd. Device aa38
07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
Subsystem: ASUSTeK Computer Inc. Device 8367
Kernel driver in use: r8169
08:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
Subsystem: ASUSTeK Computer Inc. Device 8367
Kernel driver in use: r8169
ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05)
Subsystem: Intel Corporation Device 8086
ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05)
Subsystem: Intel Corporation Device 8086
ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05)
Subsystem: Intel Corporation Device 8086
ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05)
Subsystem: Intel Corporation Device 8086
ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05)
Subsystem: Intel Corporation Device 8086
ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05)
Subsystem: Intel Corporation Device 8086
ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
The kernel BUG:
BUG: unable to handle kernel NULL pointer dereference at 00000000000001a4
IP: [<ffffffffa02e9e54>] edac_create_mci_instance_attributes+0xe0/0x139 [edac_core]
PGD 6393ff067 PUD 6393fe067 PMD 0
Oops: 0000 [#1] SMP
last sysfs file: /sys/devices/pci0000:ff/0000:ff:02.1/uevent
CPU 1
Modules linked in: i7core_edac(+) tpm cdc_acm wmi soundcore processor serio_raw evdev tpm_bios i2c_i801 asus_atk0110 pcspkr edac_core button shpchp pci_hotplug snd_page_alloc i2c_core sha256_generic aes_x86_64 aes_generic cbc ext4 mbcache jbd2 crc16 dm_crypt dm_mod raid1 md_mod sg sr_mod sd_mod cdrom crc_t10dif usbhid hid usb_storage uhci_hcd ahci libahci libata scsi_mod ehci_hcd usbcore r8169 mii thermal thermal_sys nls_base [last unloaded: scsi_wait_scan]
Pid: 705, comm: modprobe Tainted: G W 2.6.36-rc6-amd64 #1 P6T6 WS REVOLUTION/System Product Name
RIP: 0010:[<ffffffffa02e9e54>] [<ffffffffa02e9e54>] edac_create_mci_instance_attributes+0xe0/0x139 [edac_core]
RSP: 0018:ffff880638f37b88 EFLAGS: 00010282
RAX: 00000000000001a4 RBX: ffff88063c718a20 RCX: 0000000000000000
RDX: ffff88063a56c0f0 RSI: ffffffffa02ecdc0 RDI: ffff88063c718a30
RBP: ffffffffa04bcd28 R08: ffff880001a31928 R09: ffff88063a56c130
R10: ffff88063c718370 R11: ffff88063a56c130 R12: ffff88063c718a30
R13: ffff88063a56c000 R14: ffff88063a56c130 R15: 0000000000000060
FS: 00007f57294e1700(0000) GS:ffff880001a20000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000001a4 CR3: 00000006393fc000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process modprobe (pid: 705, threadinfo ffff880638f36000, task ffff880639209b40)
Stack:
ffff88063a56c0f0 ffff88063c718370 ffff8806399802d0 ffff88063c718360
<0> ffffffffa04bcc38 ffff88063c718370 ffff88063a56c000 ffff88063a56c130
<0> 0000000000000060 ffffffffa02e9e75 ffff88063a56c0f0 ffff88063a56c0f0
Call Trace:
[<ffffffffa02e9e75>] ? edac_create_mci_instance_attributes+0x101/0x139 [edac_core]
[<ffffffffa02e9f1b>] ? edac_create_sysfs_mci_device+0x6e/0x1d3 [edac_core]
[<ffffffffa02e89a5>] ? edac_mc_add_mc+0x8d/0x16d [edac_core]
[<ffffffffa04bbdd8>] ? i7core_probe+0x842/0x9f9 [i7core_edac]
[<ffffffff811a24e7>] ? local_pci_probe+0x49/0x93
[<ffffffff811a3226>] ? pci_device_probe+0xc2/0xef
[<ffffffff81228782>] ? driver_sysfs_add+0x66/0x8d
[<ffffffff812288c3>] ? driver_probe_device+0xa8/0x138
[<ffffffff812289a2>] ? __driver_attach+0x4f/0x6f
[<ffffffff81228953>] ? __driver_attach+0x0/0x6f
[<ffffffff81227f4c>] ? bus_for_each_dev+0x44/0x78
[<ffffffff812283a4>] ? bus_add_driver+0xa8/0x1f0
[<ffffffff81228c49>] ? driver_register+0x90/0xf8
[<ffffffff811a3470>] ? __pci_register_driver+0x4e/0xbe
[<ffffffffa04c007d>] ? i7core_init+0x7d/0x9d [i7core_edac]
[<ffffffffa04c0000>] ? i7core_init+0x0/0x9d [i7core_edac]
[<ffffffff81002079>] ? do_one_initcall+0x78/0x131
[<ffffffff81072982>] ? sys_init_module+0x97/0x1d5
[<ffffffff81008a02>] ? system_call_fastpath+0x16/0x1b
Code: 8d 63 10 4c 89 33 48 c7 c6 c0 cd 2e a0 4c 89 e7 48 89 43 08 48 89 18 48 8b 45 10 4c 89 6b 58 48 89 43 50 48 8b 45 10 48 8b 14 24 <48> 8b 08 31 c0 e8 8e 33 ea e0 85 c0 75 3c 48 8b 43 50 4c 89 e2
RIP [<ffffffffa02e9e54>] edac_create_mci_instance_attributes+0xe0/0x139 [edac_core]
RSP <ffff880638f37b88>
CR2: 00000000000001a4
---[ end trace a7919e7f17c0a727 ]---
--
.''`. martin f. krafft <madduck@d.o> Related projects:
: :' : proud Debian developer http://debiansystem.info
`. `'` http://people.debian.org/~madduck http://vcs-pkg.org
`- Debian - when you have better things to do than fixing systems
"when faced with a new problem, the wise algorithmist
will first attempt to classify it as np-complete.
this will avoid many tears and tantrums as
algorithm after algorithm fails."
-- g. niruta
Attachment:
digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)