[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: distributed batch processing

Just a few things to clarify:

With SGE 5.3, the FIFO is the default scheduling
policy. However, you can configure the scheduler to
run in SGE EE (Enterprise Edition) mode, which has the
4 additional scheduling policies (deadline, share
tree, override, and functional). With SGE 6.0, you get
resource reservation and urgency. There are also
"projects" and "departments" that you can setup to
group the jobs.

All of those are opensource (even the EE mode) and can
be downloaded from the SGE homepage:


IMO, Maui is better for backfilling parallel jobs, but
SGE is better for the "compute farm" type of

Also, the "array job" functionality parameterizes the
job group. To run 200 tasks, all you need to do is a

% qsub -t 1-200 <job script>

And inside the jobscript:
foo -i file$SGE_TASK_ID.data -p 0.1

And you can monitor all the subtasks as one job.

Another advantage is that SGE offers failover
functionality, which allows you to have one or more
standby masters.


Discover Yahoo! 
Have fun online with music videos, cool games, IM and more. Check it out! 

Reply to: