Re: multi-TB diskarrays ???
On Thu, 2002-11-07 at 18:47, Michael D. Schleif wrote:
>
> "Michael D. Schleif" wrote:
> >
> > Having never done this with linux, I'm asking this at the lowest
> > possible level to facilitate very exhaustive research:
> >
> > What do I need to know to design a debian fileserver attached to a 4-5TB
> > diskarray?
>
> Yes, I am vague with this request.
>
> Fortunately, we are in a position to specify a very large ``fileserver''
> for storing a large and growing quantity of image files.
>
> Since we are in position to specify this beast, I do not want to taint
> alternatives with any predisposition. My first vision is of a large and
> complex nfs server, filesystems of which are to be mounted by several
> medical imaging systems spread over a small campus of buildings
> connected by GB fiber. This is not carved in stone; but, delivers a
> glimpse of our task at hand.
>
> Basically, I need to do considerable research regarding maximum
> filesize, maximum volume size, &c. Pointers to these issues are
> welcome.
Since it's a medical imaging repository (that has all *sorts* of
legal pitfalls), I'd not even think about IDE...
Lets see: 4TB, using 180GB disks gives us ceil(4096GB/180GB) = 23
disks.
Given that you'll want hot-swapping RAID5 redundancy, let's bump
that up to 29 disks. (You'll want one on-site just in case a disk
pukes.)
That's 5 "half-shelves", rounded to 3 shelves.
Normally, you don't want to put more than 3 disks per SCSI controller,
thus 10 SCSI controllers!!
Then again, will this be mostly (1) write the image, (2) read it
2 or 3 times, then (3) store it for 7 years? In that case, loading
the UW-SCSI3 controllers may just work, and then just 3 controllers
would suffice. Of course, the more the merrier!! And if a controller
does die, it minimizes the damage...
Don't forget dedicated controller(s) for (parallel) tape backups.
(Here's an idea: now that *cheap* 320GB IDE drives are out, backup
your images [via NFS] to a separate server packed with IDE drives...)
We have a tape silo that has ~100 SuperDLT tapes and 6 tape drives.
All is bar-coded, automated, etc. Works with Alpha/VMS, but don't
know if there's a Linux driver. This is the kind of hardware that
you'll need for such disk farm.
How many PCI slots to mid- or top-end x86 Compaqs & IBMs have?
How much guaranteed uptime do you need? If it pukes at 10AM on a
Monday, are you polishing your resume, and a dozen people need to
get re-X-rayed, CAT scanned and MRIed? If so, I'd (really) look
at Alpha/VMS or Tru64 for *BULLET-PROOF* filesystem clustering.
Shoot (with a big pistol) one of the computers, and that other one
doesn't bat an eye, and file integrity is maintained.
Also, the clustering software allows the sysadmin to present the
cluster to the world as a single node, and load-balances incoming
IP connections. Thus, when you add a node (SCSI-clustering allows
for 4 nodes), or when the aforementioned node gets shot, it is
transparent to the world.
Of course, you'll pay big bucks to HP/Compaq/DEC (mainly for disks,
controllers, licenses, etc). The computers (probably DS10s) will
be the least of your costs.
Once it's all implemented, I'd love to hear how much you spent
on it. If it's less than $75K, I'll be really surprised.
Lastly, remember that file locking in NFS sucks, so you'd have to
watch out for that...
--
+------------------------------------------------------------+
| Ron Johnson, Jr. mailto:ron.l.johnson@cox.net |
| Jefferson, LA USA http://members.cox.net/ron.l.johnson |
| |
| "they love our milk and honey, but preach about another |
| way of living" |
| Merle Haggard, "The Fighting Side Of Me" |
+------------------------------------------------------------+
Reply to: