[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: (Large) Logical Volume Management

--On Tuesday, March 23, 2004 11:13 +0100 Samuele Catusian <samuele.catusian@studenti.ing.unipi.it> wrote:

Hail folks.

I've to set up an ~1TB SAN on our network. We're thinking about recycling
an existing server with an HP SmartArray 641 RAID Controller and
expanding  it with another SA641 controller and some more disks, or
directly  purchasing an HP fiber Storage Area Network.
In both cases I'll have to create a unique logical volume, and I'm
wondering which logical volume manager to use. The machine will be a
production system, it must be stable and reliable, fairly fast in disks
access, and I'd like to run a 2.6 kernel on it. Lately I've used EVMS on
some small systems and it left me well impressed; is it sufficiently
mature and stable to be used with good results on such a system? Are
there  other _valid_ alternatives?

I use native LVM, just skipping the EVMS abstraction bit, gain a bit of performance.

And, of course, I'll have to use a journaled filesystem on top of the
LVM. The average size of the files is about hundreds KiloBytes, seldom
reaching the whole MB. The directories hierarchy will be fixed and highly
structured, organized like this:


The number of stored files will be about 1,5 millions, and the estimated
access rate will remain lower than 1,000 access/sec, with 30% write and
70% read.
I've played for so long with ext3 and XFS filesystems, but both seems to
have efficiency problems with setups like this. May someone give me some
advices about the filesystem choice? Could ReiserFS be a valid solution?
Should I consider other filesystems?

ReiserFS is a good, and stable choice with late-model 2.4 series kernels. 2.6 is not yet production ready. ReiserFS also allows for online/hot expansion, only really supposed to be for the brave but seems to work fine, no data loss for me but YMMV!

Reiser scales well under all sorts of loads but really kicks the crap out of everything in small files. XFS is not a choice, I can give you a tome of XFS horror stories if you like. Performance is abysmal for anything but sequential I/O, but there it does shine VERY well. Random and more real-world I/O loads it just seems to pig out. Also if you happen to rm a LOT of files at once and fill up the metadata jounral before it all gets flushed it will either corrupt your filesystem, or just forcefully unmount it. XFS seems to be like a house of cards....very easy to upset, and it's only recourse is to airbag when it gets into any kind of trouble. All too often I've had to run xfs_repair, losing all benefits of the journal. Also quota information can only be rebuilt before the volume is mounted r/w.

Michael Loftis
Modwest Sr. Systems Administrator
Powerful, Affordable Web Hosting

Reply to: