High I/O wait times in kernels since 2.6.18
Hello,
i have problems with extremely high IO wait times for some operations
like apt-get-update and apt-get upgrade. Even checking the disks with
hdparm shows this problem. I have not seen this problem with the I/O
wait until recent kernels.
The system disk is rather slow:
hdparm -t /dev/hda
/dev/hda:
Timing buffered disk reads: 26 MB in 3.23 seconds = 8.06 MB/sec
Other disks are faster:
hdparm -t /dev/hdb
/dev/hdb:
Timing buffered disk reads: 70 MB in 3.04 seconds = 23.05 MB/sec
but do also show the extremely long wait times, up to 97% wait and only
a few % CPU. The disks have DMA enabled and I have checked the memory
for a long time with no problems.
The problem can be related to the latency bug reported for recent
kernels: "Large I/O operations result in slow performance and high
iowait times", http://bugzilla.kernel.org/show_bug.cgi?id=12309
but running the provided programs ProcessSchedulerTest and
ThreadSchedulerTest does not show the problems.
I have tried three CPU schedulers: cfq (default), anticipatory and
deadline with small differences. Kernels tried recently are
2.6.18-6-686, 2.6.25-2-682 and 2.6.26-1-686.
Programs running consuming most of the memory are evolution and galeon.
However, shutting them down does not change anything.
dmesg does not show anything strange, except maybe:
1) Timer
[ 0.154492] * Found PM-Timer Bug on the chipset. Due to workarounds
for a bug,
[ 0.154495] * this clock source is slow. Consider trying other clock
sources
2) CPU
[ 0.112943] weird, boot CPU (#0) not listed by the BIOS.
[ 0.084928] CPU: L1 I cache: 16K, L1 D cache: 16K
[ 0.084994] CPU: L2 cache: 256K
[ 0.085036] Intel machine check architecture supported.
[ 0.085079] Intel machine check reporting enabled on CPU#0.
3) HDD
[ 2.804171] hda: IBM-DTTA-351010, ATA DISK drive
[ 2.808019] Marking TSC unstable due to: TSC halts in idle.
[ 3.142742] hda: host max PIO4 wanted PIO255(auto-tune) selected PIO4
[ 3.143077] hda: UDMA/33 mode selected
[ 19.415943] hda: max request size: 128KiB
[ 19.426396] hda: 19807200 sectors (10141 MB) w/466KiB Cache,
CHS=19650/16/63
[ 19.426502] hda: cache flushes not supported
[ 19.426629] hda: hda1 hda2 hda3 hda4
Comparing this with an old boot log for 2.6.14-1-686 shows:
1) Timer
Using tsc for high-res timesource
Real Time Clock Driver v1.12
2) CPU
CPU: After generic identify, caps: 0383f9ff 00000000 00000000 00000000
00000000 00000000 00000000
CPU: After vendor identify, caps: 0383f9ff 00000000 00000000 00000000
00000000 00000000 00000000
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 256K
CPU: After all inits, caps: 0383f9ff 00000000 00000000 00000040 00000000
00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
mtrr: v2.0 (20020519)
CPU: Intel(R) Celeron(TM) CPU 1400MHz stepping 04
3)
hda: IBM-DTTA-351010, ATA DISK drive
hda: max request size: 128KiB
hda: 19807200 sectors (10141 MB) w/466KiB Cache, CHS=19650/16/63,
UDMA(33)
hda: cache flushes not supported
hda: hda1 hda2 hda3 hda4
For 2.6.26-2-686 we have
cat /proc/mtrr
reg00: base=0x00000000 ( 0MB), size= 256MB: write-back, count=1
reg01: base=0x10000000 ( 256MB), size= 128MB: write-back, count=1
reg02: base=0xe8000000 (3712MB), size= 64MB: write-combining, count=1
reg03: base=0xd8000000 (3456MB), size= 128MB: write-combining, count=2
and
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 11
model name : Intel(R) Celeron(TM) CPU 1400MHz
stepping : 4
cpu MHz : 1567.779
cache size : 256 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat
pse36 mmx fxsr sse up
bogomips : 3137.89
clflush size : 32
power management:
Thank you for your help,
Svante Signell
Reply to: