Re: cluster performance
On Sat, Sep 18, 2004 at 10:02:36PM +0800, Abd Ghanie Abd Aziz wrote:
> Dear all,
>
> i'm new to this community. i hope u all can help me about beowulf cluster,
> for your information, i already have mini cluster consist of one server
> (single xeon cpu) and 8 nodes ( every nodes have 1 p4 cpu).. everything
> works fined except the connection.. i've seen that my job running only at
> 50%++ of the cpu power on every nodes.. how can i improve my cluster?
I'm guessing it's MPI based on what you're talking about below... In that
case, maybe your code is blocked waiting for data from other machines. A
lower-latency (or higher bandwidth, if that's the problem) network will help
get the most out of your CPUs. If it's just latency, maybe running two jobs
at once on your cluster would give the CPU something to run while one of the
jobs is waiting for an MPI message. If you can re-write the code to send
data earlier, and/or hold off on using it until it's already computed
everything it can without the data from other nodes, that will improve
overlapping communication and computation.
You need to know what your job does, and then you can figure out what will
benefit it.
> i've
> read that they ask to incread the P4_SOCKBUFSIZE but i dont actually
> understand how to do it.... below is the steps... i already did the first
> step but for the second one i havent do it yet... i dont know.. i'm using
> bash... can any body help me how to do the second step in bash??? what is
> the command actually.. i know this is the stupid question but sorry... i'm
> new to linux.. i hope someone can help me... i hope this thing will work
> for me to increase my cluster performance....
>
> 1) At the OS level, you need to zap a couple of values that determine the
> upper limit allowed for socket buffers. This is, in my opinion, best done
> by adding the following two lines to the system file /etc/rc.d/rc.local (as
> root, of course), and rebooting:
>
> echo 1048576 > /proc/sys/net/core/rmem_max
> echo 1048576 > /proc/sys/net/core/wmem_max
Debian doesn't have rc.local. Put those commands in a file
e.g. /etc/init.d/tune-mpi.sh, and make it executable. run
update-rc.d tune-mpi defaults
Then run the shell script. things to google: chmod, shell script
> This sets the upper limit on socket buffers to 1 MB.
>
> 2) At the level of mpi, you need to make the following entry in your .cshrc
> (or the equivalent commands in .bashrc, if you bash):
>
> setenv P4_SOCKBUFSIZE 524288
So edit your .bashrc (or .bash_profile), and add a line that says:
export P4_SOCKBUFSIZE=524288
Run emacs .bashrc and use the drop-down menus if you don't know the key
commands (like CTRL-X CTRL-C to quit :)
things to google: bash profile bashrc
--
#define X(x,y) x##y
Peter Cordes ; e-mail: X(peter@cor , des.ca)
"The gods confound the man who first found out how to distinguish the hours!
Confound him, too, who in this place set up a sundial, to cut and hack
my day so wretchedly into small pieces!" -- Plautus, 200 BC
Reply to: