Re: Building a cluster with debian?
On Wed, 3 Dec 2008 12:40:45 +0200
Johann Spies <jspies@sun.ac.za> wrote:
> On Tue, Dec 02, 2008 at 04:53:45PM +1100, Alex Samad wrote:
>
> > I believe there is software out there that will do that, I read about
> > some recently, but can't find reference to it now :(
>
> Have a look at http://gridengine.sunsource.net/
>
Thanks, looks like a glorified pbs mostly but may be of some help. I was hoping
to somehow abuse numa and processor queues to make the cluster look as a single
numa computer but that seems to require hardware support (multi processor numa
system and not several systems combined into one numa).
The solutions I found so far:
1. Hardware - Buy a very expensive multi opetron system (not an option at this time)
2. ssi (single system image) cluster, seems to be what I want, it's just not
clear to me at the moment if there is anything that can properly migrate threads
http://en.wikipedia.org/wiki/Single-system_image
3. Batch/Grid systems
Your reference seems to fall into the third category.
The problem with this is that you either need to use mpi or cluster openmp
(which I haven't managed to get working on non-opetron systems) or to run
several unconnected jobs in parallel.
I prefered number 2 but as I said, it doesn't seem to handle multi threaded and
shared memory programs, most of them not even as a whole (i.e once you have
threads the whole job will not migrate).
I want to get more threads for matlab. The solution is matlab
distributed computing (which costs about 4000$, not cheap also), and agian,
mostly allows managing several jobs in parallel on a cluster but not a
massively parallel job
I will read a bit more into these as the machines are supposed to arrive
tomorrow finally (4 quad core machines). Will try to give a followup if anyone
is interested.
> Regards
> Johann
>
>
Reply to: