On Tue, 2004-11-16 at 15:21 -0500, Ben Russo wrote: > (see below for long story background ) > > The last time I created a large HW RAID5 volume (1.6 TB) the kernel was > unable to see all of it... If I create several smaller block devices > (like 400GB each) can LVM bind them together into a larger single > filesystem? ( I am aiming for 4-6 TB ) Which kernel? 2.6 should be able to do this. http://www.kniggit.net/wwol26.html "Linux 2.6 will include improved 64-bit support on block devices that support it, even on 32-bit platforms such as i386. This allows for filesystems up to 16TB on common hardware." Of course, ext2/3 wouldn't be my 1st choice of filesystem, either. XFS, probably, since it was designed for huge files. See below, for a final comment. > Is there anyway to greate a BIG robust random rw access filesystem that > is transparently compressed and supports large files (up to several > gigabytes each? ) > > I have seen cramfs and squashfs both of which have sub 2GB filesize [snip] > is yes, but only if it is CHEAP. This raw billing data compresses > at a rate of 20:1 or 30:1 is just fixed records, most of which is > just ascii spaces or ascii 0-9. So if I could find a compressed > filesystem solution that would handle this it would be great, I > could get an IDE or SATA disk array with hardware RAID 5 and have > a huge (4 or 6 TB ) filesystem and with transparent compressed > filesystem (even if it was only 5:1) that would be enough for > several years worth of data. Even if it is very slow disk I/O, it > would still be faster than offsite tape. And it would be immensely > easier! Instead of a compressed fs, why not use a program like rzip? 2048MB/30 = a paltry 68MB rzip was designed to compress Really Large Files. Slap a few 300GB drives in a little server, and you could fit almost 13,000 such compressed files (an estimated 2,600 days, [7 years] worth) in such a box. -- ----------------------------------------------------------------- Ron Johnson, Jr. Jefferson, LA USA PGP Key ID 8834C06B "Never for the sake of peace and quiet deny your convictions." Dag Hammarskjold
Attachment:
signature.asc
Description: This is a digitally signed message part