Re: Huge AMD Opteron cluster installed with FAI and Debian amd64
Greetings!
"Matthias H. Hennig" <hennig@cn.stir.ac.uk> writes:
> Hi,
>
> this discussion is interesting for me, as we're just about to update our
> cluster. So far, we're using the messy setup Camm just described - and
> this is no longer working reliably, as it's still based on an heavily
> upgraded woody (we need gcc-3 among other things).
>
> For that reason we have decided to go for sarge and want to get rid of
> the problem keeping the nfsroots (/etc, /var and all the rest) of the
> clients up to date after every upgrade.
>
> As I understand, FAI can handle this - what is your experience? And how
> does it work precisely, simply by chroot or some other method?
> The problem we're facing is that most nodes are pc's with different
> hardware configuration, so almost each node needs a custom kernel as
> well. Is this supported well in FAI?
>
Am interested in this too.
> Btw. a nice way to increase the size of your cluster is simply to ask
> the technicians for old hardware (which means everything >600MHz) that
> is no longer used. Usually, these computers are from student labs and
> would otherwise go to charity (which is of course fine as well...).
>
Am particularly interested how you make use of a non-uniform cluster.
Most of the applications (MPI) we've developed will run as fast as the
slowest node.
Take care,
> Matthias
>
>
> On Tue, 2004-11-16 at 15:13, Camm Maguire wrote:
> > Thank you for your work on this! Looks quite impressive -- would like
> > to try it one day. Fortunately or not, due to Debian's install-once
> > ease of maintenance, we're still running the same many-times
> > apt-get-upgraded local disk installation installed on our cluster far
> > before any of these nicer tools became available. What a nice
> > 'problem' to have with Debian!
> >
> > Take care,
> >
> > Thomas Lange <lange@informatik.uni-koeln.de> writes:
> >
> > > Hi,
> > >
> > > another huge and fast cluster was installed using FAI.
> > > http://www.informatik.uni-koeln.de/fai/
> > >
> > > The HPC2N Opteron cluster named sarek (after Sarek National Park) has
> > > a total of 384 processors and 1.54 Tbyte of memory:
> > >
> > > * 190 HP DL145 nodes, with dual AMD Opteron 248 (2.2GHz)
> > > * 2 HP DL585, with dual AMD Opteron 248 (2.2GHz)
> > > * 8 GB memory per node
> > > * Myrinet 2000 high speed interconnect
> > >
> > > It's running Linux 2.6.x and is using the Debian Pure64 port. This
> > > cluster was listed at rank 168 on the top500.org list of june 2004 and
> > > is now at rank 220 (11/2004). It's homepage is
> > >
> > > http://www.hpc2n.umu.se/resources/sarek.html
> > >
> > > Thanks to Bjorn Torkelsson who pointed me to this nice FAI example.
> > >
> > > --
> > > regards Thomas
> > >
> > >
> > > --
> > > To UNSUBSCRIBE, email to debian-beowulf-REQUEST@lists.debian.org
> > > with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> > >
> > >
> > >
> > >
> >
> > --
> > Camm Maguire camm@enhanced.com
> > ==========================================================================
> > "The earth is but one country, and mankind its citizens." -- Baha'u'llah
> >
>
>
> --
> To UNSUBSCRIBE, email to debian-beowulf-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
>
>
>
>
--
Camm Maguire camm@enhanced.com
==========================================================================
"The earth is but one country, and mankind its citizens." -- Baha'u'llah
Reply to: