Bug#600031: updating kernel causes slow write speed to raid
Ben
That has solved it. Now getting around 27MB/s write on ext4 and 97MB/s
read with the experimental kernel. I'm happy with these speeds on this
hardware. It is still a little slower then with windows but I can live
with that.
May be able to get better by tuning read-ahead buffer and
vm.dirty_ratio. Do you think there is any merit in adjusting these?
Thanks
Dave
On 14 October 2010 03:46, Ben Hutchings <ben@decadent.org.uk> wrote:
> On Thu, 2010-10-14 at 00:31 +0100, Dave Clarke wrote:
>> Running bonnie++ locally. Have formatted / as ext3 and /home/ as ext4.
>> using raid5. Also to note that when booted with windows server 2003
>> using Everest disk benchmark it does an average of 34MB/s (peaking to
>> 50MB/s).
>>
>> Attached are bonnie++ output for the two kernel versions and ext3 and
>> ext4 (ext4 won't mount with 2.6.26 kernel).
>> And meminfo for both kernels taken when the system is loaded and unloaded.
>
> OK, it looks like it's not so short of memory. And this is really quite
> poor write performance.
>
> Could you also test Linux 2.6.36-rc6, as packaged in experimental?
>
> Ben.
>
> --
> Ben Hutchings
> Once a job is fouled up, anything done to improve it makes it worse.
>
== Loaded (bonnie++) ==
MemTotal: 514348 kB
MemFree: 5432 kB
Buffers: 512 kB
Cached: 464020 kB
SwapCached: 148 kB
Active: 250484 kB
Inactive: 239976 kB
Active(anon): 13300 kB
Inactive(anon): 13588 kB
Active(file): 237184 kB
Inactive(file): 226388 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 514348 kB
LowFree: 5432 kB
SwapTotal: 1508348 kB
SwapFree: 1506756 kB
Dirty: 28 kB
Writeback: 0 kB
AnonPages: 25824 kB
Mapped: 3584 kB
Shmem: 916 kB
Slab: 11876 kB
SReclaimable: 5924 kB
SUnreclaim: 5952 kB
KernelStack: 840 kB
PageTables: 1076 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 1765520 kB
Committed_AS: 166828 kB
VmallocTotal: 508152 kB
VmallocUsed: 7124 kB
VmallocChunk: 486968 kB
HardwareCorrupted: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 4096 kB
DirectMap4k: 16128 kB
DirectMap4M: 507904 kB
== Unloaded ==
MemTotal: 514348 kB
MemFree: 487468 kB
Buffers: 948 kB
Cached: 6584 kB
SwapCached: 1952 kB
Active: 3992 kB
Inactive: 7680 kB
Active(anon): 2084 kB
Inactive(anon): 2444 kB
Active(file): 1908 kB
Inactive(file): 5236 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 514348 kB
LowFree: 487468 kB
SwapTotal: 1508348 kB
SwapFree: 1481796 kB
Dirty: 40 kB
Writeback: 0 kB
AnonPages: 2620 kB
Mapped: 2636 kB
Shmem: 380 kB
Slab: 8312 kB
SReclaimable: 2220 kB
SUnreclaim: 6092 kB
KernelStack: 848 kB
PageTables: 1088 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 1765520 kB
Committed_AS: 168020 kB
VmallocTotal: 508152 kB
VmallocUsed: 7124 kB
VmallocChunk: 486968 kB
HardwareCorrupted: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 4096 kB
DirectMap4k: 16128 kB
DirectMap4M: 507904 kB
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 384 92 29431 20 8050 5 1327 97 82875 19 514.1 29
Latency 296ms 1322ms 1178ms 52896us 58501us 1105ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 3980 17 +++++ +++ 7072 14 8390 34 +++++ +++ 11415 23
Latency 55696us 1290us 1610us 479us 77us 273us
1.96,1.96,heterogeneous,1,1287064582,1G,,384,92,29431,20,8050,5,1327,97,82875,19,514.1,29,16,,,,,3980,17,+++++,+++,7072,14,8390,34,+++++,+++,11415,23,296ms,1322ms,1178ms,52896us,58501us,1105ms,55696us,1290us,1610us,479us,77us,273us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 379 93 28483 17 8018 5 1922 98 86721 19 490.3 26
Latency 338ms 520ms 1841ms 16485us 94116us 955ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 7074 30 +++++ +++ 10515 20 8275 34 +++++ +++ 10228 21
Latency 55852us 1312us 1393us 364us 30us 80us
1.96,1.96,heterogeneous,1,1287064964,1G,,379,93,28483,17,8018,5,1922,98,86721,19,490.3,26,16,,,,,7074,30,+++++,+++,10515,20,8275,34,+++++,+++,10228,21,338ms,520ms,1841ms,16485us,94116us,955ms,55852us,1312us,1393us,364us,30us,80us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 367 91 23711 14 8168 4 1799 96 87187 19 477.5 26
Latency 484ms 823ms 1177ms 54329us 21160us 951ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 5329 23 +++++ +++ 10963 21 8925 36 +++++ +++ 10717 21
Latency 43286us 1658us 1575us 305us 34us 272us
1.96,1.96,heterogeneous,1,1287062329,1G,,367,91,23711,14,8168,4,1799,96,87187,19,477.5,26,16,,,,,5329,23,+++++,+++,10963,21,8925,36,+++++,+++,10717,21,484ms,823ms,1177ms,54329us,21160us,951ms,43286us,1658us,1575us,305us,34us,272us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 391 93 27932 16 8151 5 1920 99 83089 19 517.1 29
Latency 369ms 621ms 822ms 15537us 147ms 763ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 4247 18 +++++ +++ 6785 13 8191 34 +++++ +++ 11002 22
Latency 38341us 1258us 1339us 597us 69us 1504us
1.96,1.96,heterogeneous,1,1287062035,1G,,391,93,27932,16,8151,5,1920,99,83089,19,517.1,29,16,,,,,4247,18,+++++,+++,6785,13,8191,34,+++++,+++,11002,22,369ms,621ms,822ms,15537us,147ms,763ms,38341us,1258us,1339us,597us,69us,1504us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 390 92 26945 16 7908 5 1890 98 85417 19 483.5 26
Latency 409ms 622ms 1321ms 23255us 61678us 1162ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 4194 18 +++++ +++ 11135 22 8505 36 +++++ +++ 10902 21
Latency 19699us 1365us 1742us 2751us 25us 14477us
1.96,1.96,heterogeneous,1,1287062229,1G,,390,92,26945,16,7908,5,1890,98,85417,19,483.5,26,16,,,,,4194,18,+++++,+++,11135,22,8505,36,+++++,+++,10902,21,409ms,622ms,1321ms,23255us,61678us,1162ms,19699us,1365us,1742us,2751us,25us,14477us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 369 97 23204 11 8074 4 1738 94 101309 21 376.1 21
Latency 39513us 1326ms 1116ms 64813us 22433us 1364ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 4379 24 +++++ +++ 10932 33 8535 45 +++++ +++ 10231 31
Latency 2511us 1235us 1538us 599us 141us 138us
1.96,1.96,heterogeneous,1,1287063741,1G,,369,97,23204,11,8074,4,1738,94,101309,21,376.1,21,16,,,,,4379,24,+++++,+++,10932,33,8535,45,+++++,+++,10231,31,39513us,1326ms,1116ms,64813us,22433us,1364ms,2511us,1235us,1538us,599us,141us,138us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 329 96 27473 13 9073 5 1896 98 97501 21 461.9 27
Latency 41350us 623ms 1627ms 16520us 19331us 200ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 7326 41 +++++ +++ 10650 32 8252 46 +++++ +++ 10123 33
Latency 6626us 1250us 1347us 645us 226us 1811us
1.96,1.96,heterogeneous,1,1287064331,1G,,329,96,27473,13,9073,5,1896,98,97501,21,461.9,27,16,,,,,7326,41,+++++,+++,10650,32,8252,46,+++++,+++,10123,33,41350us,623ms,1627ms,16520us,19331us,200ms,6626us,1250us,1347us,645us,226us,1811us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 379 94 27496 13 9013 5 1640 85 92182 20 470.3 27
Latency 40219us 624ms 1796ms 19854us 47974us 979ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 6513 35 +++++ +++ 10303 32 8399 44 +++++ +++ 10335 32
Latency 2274us 1218us 1288us 1343us 29us 1409us
1.96,1.96,heterogeneous,1,1287064142,1G,,379,94,27496,13,9013,5,1640,85,92182,20,470.3,27,16,,,,,6513,35,+++++,+++,10303,32,8399,44,+++++,+++,10335,32,40219us,624ms,1796ms,19854us,47974us,979ms,2274us,1218us,1288us,1343us,29us,1409us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 327 95 27880 12 8140 4 1888 98 96370 20 421.8 24
Latency 29700us 621ms 1984ms 15374us 57868us 1193ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 7051 39 +++++ +++ 10193 30 8064 44 +++++ +++ 9913 30
Latency 34400us 1230us 1340us 852us 31us 144us
1.96,1.96,heterogeneous,1,1287065199,1G,,327,95,27880,12,8140,4,1888,98,96370,20,421.8,24,16,,,,,7051,39,+++++,+++,10193,30,8064,44,+++++,+++,9913,30,29700us,621ms,1984ms,15374us,57868us,1193ms,34400us,1230us,1340us,852us,31us,144us
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
heterogeneous 1G 333 94 27602 12 9353 6 1787 92 93237 23 462.0 27
Latency 28100us 722ms 1675ms 72005us 18851us 196ms
Version 1.96 ------Sequential Create------ --------Random Create--------
heterogeneous -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 6342 34 +++++ +++ 10734 31 8343 44 +++++ +++ 10683 33
Latency 4184us 1268us 1447us 1549us 37us 1349us
1.96,1.96,heterogeneous,1,1287065440,1G,,333,94,27602,12,9353,6,1787,92,93237,23,462.0,27,16,,,,,6342,34,+++++,+++,10734,31,8343,44,+++++,+++,10683,33,28100us,722ms,1675ms,72005us,18851us,196ms,4184us,1268us,1447us,1549us,37us,1349us
Reply to: