Hardware vs Software RAID 10 performance?
(First of all, I apologise if anyone sees this twice - I first posted to
the AMD64 list, but then thought that the more general debian users list
might get a broader response)...
I'm curious as to whether anyone has experience of software RAID in
Linux giving better overall performance on RAID10 than a RAID card such
as the Adaptec 2015S.
Server: Debian Etch AMD64 on Dual Opteron 265, 1.8GHz, i.e. 4 cores
total, 4GB RAM, 4x10k Fujitsu SCSI 73GB, Adaptec 2015S zero-channel RAID
card.
Currently I have the four drives in RAID10 using the Adaptec, i.e. it
appears as one big drive to Linux. Then a few days ago I saw this video
presentation on scaling from one of the guys who developed YouTube:
http://video.google.com/videoplay?docid=-6304964351441328559
One thing that really caught my attention was around the 34:50 mark. He
talks about how they were seeing IO wait on their RAID 10 setup, which
had 5 sets of 2 disk RAID1 arrays, appearing to Linux as one device.
They thought that since Linux was only seeing one big single RAID 10
device, maybe the kernel wasn't parallelising disk reads and writes
aggressively enough. So they changed it to still use hardware RAID 1 on
the disk pairs, but then exposed these RAID 1 devices to the linux
kernel (i.e. now 5 distinct RAID 1 arrays) and then used software RAID 0
to stripe across those. This seems really simple and ingenious, since it
allows linux now to realize that there is more potential to parallelize
io. They apparently saw 20-30% increase in performance on the same
hardware after doing that.
I only have four disks, but now I'm wondering if I might see improvement
in performance by reconfiguring as two RAID 1 arrays and then doing
software RAID 0 across those.
Or, why not go even further and do JBOD on the RAID controller, and do
all the RAID10 in Linux? Would that see better performance?
I have little knowledge of what "zero channel" really means, or how good
the hardware raid processor is in the Adaptec 2015S. What I am fairly
sure of is that with the 4 Opteron cores, I have CPU cycles to spare; on
this server I am much more likely to run out of io throughput before I
run out of pure CPU cycles. So I am interested in maximizing the io,
even if it means doing a little more work in the kernel. I'm not doing
RAID5, so there's no parity work being done; I don't have a handle on
how much work RAID10 takes for software RAID. Also, really have no take
on where the real IO bottleneck is here - if putting the RAID into Linux
would actually have any benefit with this particular type of card. I
mean, if the actual bottleneck is actually on some bus or other in
between the disks and the motherboard, then it really doesn't matter
where the RAID is handled.
Does anybody have any experience of this, or wisdom to impart? I have
read all the arguments in favor of software RAID, and that's all very
nice, but what I am primarily wondering about here is if it's likely
that shifting over to software RAID might allow Linux to improve IO by
more parallelization of reads/writes, or if it's just shifting
complexity from one place to another, with the same bottleneck in
between? (In which case, I guess, might as well go with software RAID,
since it seems easier to recover from a controller crash).
Thanks!
/Neil
Reply to: