Re: HPC with Beoqulf

[Carsten Aulbert <carsten.aulbert@aei.mpg.de>]

Hi Patrick,

On Monday 17 January 2011 09:24:47 Patrick Schmid wrote:
> It has to be a HPC cause it's a given challenge: "Build a Linux HPC
> with ethernet"
> 

Good, and I can confirm it works and scales quite well (at least the beast we 
manage here ;))

> 
> The goal is to split up the load of computing a job to multiple nodes.
> 
> So you have a script / job you need to compute, and need a fast way to
> to that. That's typically it ;)
> My imagination is, that you upload this script to a master node, which
> splits it up and distributes the parts to its nodes.
> 
> hope you can understand :)
> 
> What do you think? Still possible to do that with Beowulf?

Sure, but it will depend how "social" your users will be, i.e. if they will 
partition the resources without fighting each other.

You can use "dsh" (distributed shell) to start jobs manually on a set of 
cluster nodes (or loop over ssh connections), or you can go the way to use a 
"real" job scheduler (PBS, OGE, Torque, Condor, ...) depending of what you 
really want/need...

You just need to grow with the expectations I guess, start small and become 
bigger of time if need be.

Cheers

Carsten