Re: about cluster in debian
Hi Bruno (I'm sharing this to the list again).
Bruno Sales wrote:
> Hi Carsten,
> it will be multi-threaded,
> they will need to access central storage areas
> low number of users (just to begin, i can grow up later)
Let's see. Depending on the money and needs I would suggest the following:
If you have very few threads running simultaneously, go for a single or
dual socket multi core system fitting your needs. E.g if you have at
most 6 threads running, buy a dual socket quad core Xeon or wait a bit
longer and buy the same from AMD. However, I strongly suggest, that you
come up with a benchmark like tool and talk to different vendors and ask
them to test the boxes. If you have some money to spend, I think it may
happen.
If you need more parallelism, you have to look into network
infrastructure. Is GB ethernet good enough for you, or do you need low
latency high bandwidth connections like Infiniband? That's something
only you can decide. Usually with more testing and looking into your
budget. Rule of thumb: After taking away money for storage, image
spending all the money on network and computers going for infiniband.
Then do the same for GBit ethernet and see how many more compute nodes
you could purchase .Then you have to evaluate which would be better
suited for your needs.
If you need some help on Gbit ethernet I might help a bit.
Storage: If you vast amounts of money to spend, look into the SUN X4500
running Solaris, although on this list I would advocate for a more
standard set-up with a 3ware 9560 or Areca 12xx RAID6 controller with a
bunch of not too cheap hardware. Usually you should be able to get
something which is quite fast and shoudl cost lss than 1000US$/TB net
storage.
Software:
We use the Debian package FAI to automatically install cluster nodes,
and since we don't have jobs running across nodes we use a simple batch
system called condor (http://www.cs.wisc.edu/condor/), but you may want
to look into openpgs, sun grid engine, torque among others.
HTH
Carsten
Reply to: