Re: "big" machines running Debian?
Ian McDonald <iam@st-andrews.ac.uk> writes:
> Goswin von Brederlow wrote:
>> lsorense@csclub.uwaterloo.ca (Lennart Sorensen) writes:
>>
>>> On Thu, Feb 26, 2009 at 08:54:11AM +1100, Alex Samad wrote:
>>>> most enterprise site don;t use 1TB size disk, if you want performance
>>>> you go spindles, there might be 8 disks (number pulled from the air -
>>>> based on raid6 + spares) behind 1TB
>>> And if you want disk space and are serving across a 1Gbit ethernet link,
>>> you don't give a damn about spindles and go for cheap abundant storage,
>>> which means SATA.
>>>
>>> Not everyone is running a database server. Some people just have files.
>>>
>>> Raid5/6 of a few SATA drives can easily saturate 1Gbit. And for a very
>>> small fraction of the cost of SAS drives.
>>
>> 1GBit is satturated by a single good disk already. 1GBit is a joke for
>> fast storage.
>>
>
> Erm, not on anything other than a sequential read (and even then, I've
> never seen a single disk that would actually sustain that across it's
> whole capacity).
A cheap SATA disk with 7200rpm sustains 80MB/s sequential read/write
on the outside and 40MB/s on the inside. An Seagate Cheetah 15K.6 is
specified to up to 171MB/s and SAS disks are more uniform between
outside and inside tracks.
> Even raid-5s of significant numbers of disks aren't enormously fast,
> especially under multiple access. hdparm informs me that the SATA 28+2
> spare raid-5 I have will read 170M a second. That would rapidly
> diminish under any sort of load.
For our Lustre filesystems we tested 16 SATA disks in an Infotrend SAS
raid enclosure. As raid6 we still get >450 MiB/s sequential writing
und >700MiB/s sequential reading. And that scales pretty well with
more enclosures and more clients.
In your case I would think the problem is your configuration. An 28
disk raid5 has a lot of stripes. That takes a lot of cache per stripe
and a lot of cpu to calculate parity. Plus the chance of 2 disks
failing before the spare disk can be synced mustbe HUGE. Have you ever
thought about making multiple smaller raids?
> The only thing we've found that'll stand up to real multiuser load
> (like a mail spool) is raid-10, and enough spindles.
Mail spool is like database access. Tons and tons of tiny read/write
requests. The only thing that counts there is seek time. And the only
raid level that improves seek time is raid1 (and the raid1 in raid10).
> We're beginning to see the requirement for 10GE on busy machines.
Don't forget that you have overhead too. If you only have 1GBit to the
storage then how is your server supposed to saturate the 1GBit to the
ouside world?
MfG
Goswin
Reply to: