Re: Huge AMD Opteron cluster installed with FAI and Debian amd64
Hi,
this discussion is interesting for me, as we're just about to update our
cluster. So far, we're using the messy setup Camm just described - and
this is no longer working reliably, as it's still based on an heavily
upgraded woody (we need gcc-3 among other things).
For that reason we have decided to go for sarge and want to get rid of
the problem keeping the nfsroots (/etc, /var and all the rest) of the
clients up to date after every upgrade.
As I understand, FAI can handle this - what is your experience? And how
does it work precisely, simply by chroot or some other method?
The problem we're facing is that most nodes are pc's with different
hardware configuration, so almost each node needs a custom kernel as
well. Is this supported well in FAI?
Btw. a nice way to increase the size of your cluster is simply to ask
the technicians for old hardware (which means everything >600MHz) that
is no longer used. Usually, these computers are from student labs and
would otherwise go to charity (which is of course fine as well...).
Matthias
On Tue, 2004-11-16 at 15:13, Camm Maguire wrote:
> Thank you for your work on this! Looks quite impressive -- would like
> to try it one day. Fortunately or not, due to Debian's install-once
> ease of maintenance, we're still running the same many-times
> apt-get-upgraded local disk installation installed on our cluster far
> before any of these nicer tools became available. What a nice
> 'problem' to have with Debian!
>
> Take care,
>
> Thomas Lange <lange@informatik.uni-koeln.de> writes:
>
> > Hi,
> >
> > another huge and fast cluster was installed using FAI.
> > http://www.informatik.uni-koeln.de/fai/
> >
> > The HPC2N Opteron cluster named sarek (after Sarek National Park) has
> > a total of 384 processors and 1.54 Tbyte of memory:
> >
> > * 190 HP DL145 nodes, with dual AMD Opteron 248 (2.2GHz)
> > * 2 HP DL585, with dual AMD Opteron 248 (2.2GHz)
> > * 8 GB memory per node
> > * Myrinet 2000 high speed interconnect
> >
> > It's running Linux 2.6.x and is using the Debian Pure64 port. This
> > cluster was listed at rank 168 on the top500.org list of june 2004 and
> > is now at rank 220 (11/2004). It's homepage is
> >
> > http://www.hpc2n.umu.se/resources/sarek.html
> >
> > Thanks to Bjorn Torkelsson who pointed me to this nice FAI example.
> >
> > --
> > regards Thomas
> >
> >
> > --
> > To UNSUBSCRIBE, email to debian-beowulf-REQUEST@lists.debian.org
> > with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> >
> >
> >
> >
>
> --
> Camm Maguire camm@enhanced.com
> ==========================================================================
> "The earth is but one country, and mankind its citizens." -- Baha'u'llah
>
Reply to: