Re: cern software
Hi Brett, et al,
On Mon, 2006-08-07 at 13:59 -0400, Brett Viren wrote:
> Hi Matt,
>
> Matt Zagrabelny <mzagrabe@d.umn.edu> writes:
>
> > My name is Matt Zagrabelny, I work as a Debian sysadmin for the
> > University of Minnesota Duluth. We are looking at setting up a Beowulf
> > Cluster for our Physics Department. They are advocating Scientific Linux
> > (RedHat derivative), I am advocating Debian.
Good for you.
Forgive me for ranting a bit here:
Scientific Linux comes it to variants - the FermiLab variant and the
CERN variant. I'm only familiar with the CERN one. Do to various
(external) hardware requirements, we have had to set up a few CERN SL (3
and 4) boxes, and let me tell you that the experience wasn't all that
nice.
First of all, the CSL is customised too much for CERN needs - that is,
there are general assumptions about having home directories on AFS
(though local users _are_ possible), and things like printing services
are geared to CERN's particular setup.
Also, you'd expect a distribution that has `scientific' in the name to
distribute `standard' scientific software, like GNU Scientific Library,
scilab, mathomatic, Maxima, GNUPlot, and so on - well, CSL doesn't (at
least not version 3 - version 4 has a bit more, but still pretty far
from Debian standards). And then I haven't even mentioned things like
PySci, ROOT, PAW, CERNLIB, GEANT (3.21 and/or 4).
One good thing about CSL is that they use APT or YUM :)
End of rant.
> > They need certain CERN software. They have pointed me to the following
> > website:
> >
> > http://lhcb-comp.web.cern.ch/lhcb-comp/Frameworks/
>
> Geant4 and ROOT are (currently unofficially) packaged for Debian.
:-)
ROOT was close to hitting `main' for etch, but some unfortunate license
problems prevented that. The upload to Debian is postponed till that
situation is remedied (upstream and I are working on it :)
The web-page quoted above is for LHCb - it's not a general LHC
experiment page.
> > They need things like GAUDI. I am wondering if anyone has any experience
> > with installing this software (on Debian), if anyone could give any hints, or any
> > general advice regarding CERN packages/frameworks/software.
`CERN packages' is very a loos term. There are many weird packages
spread out over the various experiments and departments.
> I don't think Gaudi is packaged for Debian.
I'm pretty certain it's not.
> It uses its own package
> manager, CMT (www.cmtsite.org)
CMT is an unfortunate mix of a build system and a package manager - very
much geared to SL. It may work on other GNU/Linux systems, but probably
won't work anywhere else :(
There's an unfortunate tendency, especially in the GRID context, to
distribute practically the whole OS with some custom tool. A lot of
GRID tools insists that you have a particular patched up version of say
OpenSSL (not GNU TLS -_OpenSSL_), and that tendency propagates to almost
all layers of the software. It's silly, because the distributions
(Debian, RH, SUSE, ...) are much better at distributing software than
any custom code could ever be. The distributions have years and years
of experience in doing that, and making sure everything works seamlessly
anywhere in the world - something CERN IT is notoriously bad at.
> that builds Gaudi and other LHC
> packages and provides environment variable setup.
Erhm, `LHC packages' is too loose. Not all experiments uses CMT, and
some don't use it for everything. For example, ALICE, my experiment, do
not use it. We have ROOT as our foundation, and therefor need only very
particular code - the abstraction layerr comes for free via ROOT.
> Given that, there isn't much difference in building CMT controlled
> packages on Debian than on SL or other GNU/Linux distributions.
Hopefully that's true. However, be aware of RPM hooks in CMT, and other
ill-conceived hacks that could distribut the system.
I'd suggest setting up a test machine before you plunge into the big
install. If you package CMT for Debian, perhaps that would minimise
your problem.
> Some packages that I find useful for my Debian cluster:
>
> diskless (Debian specific, manage config on worker nodes)
> usepackage (Manage environment for non-Debian packages)
> cexec (exec commands on worker nodes in parallel)
> distcc (distributed compilation)
>
> I boot my nodes from the network and their root FS are all NFS
> mounted. Local disk is currently only used for swap space. This,
> plus diskless, makes adding new nodes and upgrading software extremely
> easy.
Sounds good. Don't you start up PROOF servers too?
> I use torque/maui for the batch system. These are unfortunately not
> available as a Debian package to my knowledge.
>
On Mon, 2006-08-07 at 21:57 -0500, Dirk Eddelbuettel wrote:
> On 7 August 2006 at 13:59, Brett Viren wrote:
> | I use torque/maui for the batch system. These are unfortunately not
> | available as a Debian package to my knowledge.
>
> Incidentally, I was looking for some simple batch systems too -- nothing
> fancy at this point. Any recommendations from this list? Some pointers:
>
> -- GNU queue seems nice, but dead.
Are you sure it's dead? The page
http://savannah.gnu.org/projects/gnu-queue/
has an not so old entry.
> -- OpenPBS anyone?
OpenPBS http://www.openpbs.org/ seems to require registration etc. for
downloading. This could indicate that the license is not that good for
Debian.
> -- LLNL's slurm and munge looked interesting and simple.
> -- Any unofficial package somewhere that people have experience with?
Try apt-get.org :)
> -- Condor ?
Condor seems to be supported on `sarge' via Alien installs (though they
call it `Red Hat Debian Linux' on their page
http://www.cs.wisc.edu/condor/ - Brr.) The license seems to be BSD
like, with the advertisement clause however.
> -- Or something really simple like dsh?
There's no load balancing, resource management, or accounting with DSH -
it's simply a remote executor. Probably not good enough for any
production cluster. The author seems to use the program for things
like `apt-get update' on a bunch of machines, which makes sense - but
for job queues? - no I don't think so.
Yours,
--
___ | Christian Holm Christensen
|_| | -------------------------------------------------------------
| | Address: Sankt Hansgade 23, 1. th. Phone: (+45) 35 35 96 91
_| DK-2200 Copenhagen N Cell: (+45) 24 61 85 91
_| Denmark Office: (+45) 353 25 404
____| Email: cholm@nbi.dk Web: www.nbi.dk/~cholm
| |
Reply to: