distributed batch processing

[Paul Brossier <piem-lists@altern.org>]

Hi all,

I sent the following mail to -devel, but thought this list will
have some fine advices. I am not really willing to set up a
cluster just to distribute jobs though.

Many thanks, Paul

----- Forwarded message from Paul Brossier <piem-lists@altern.org> -----

Date: Tue, 10 May 2005 01:05:28 +0100
From: Paul Brossier <piem-lists@altern.org>
To: debian-devel@lists.debian.org
Subject: distributed batch processing
Reply-To: Paul Brossier <piem-lists@altern.org>
Resent-Date: Mon,  9 May 2005 19:05:57 -0500 (CDT)

Hi all,

I am looking at ways to distribute batch jobs on various hosts.
Essentially, i have N different command lines, and M different
hosts to run them on:

        foo -i file1.data -p 0.1
        foo -i file2.data -p 0.1
        foo -i file3.data -p 0.1
        ...
        foo -i file1.data -p 0.2
        ...

I had a try with 'queue' [1], but it seems rather obsolete now.
I am now seeking recent alternatives. I went across a few
solutions, such as DQS [2] (non-free, unmaintained), OpenPBS [3]
(non-free), and distribulator [4] (looks interesting).

Now i feel like i have missed something obvious. Is there a tool
out there that i could use as a drop in replacement for queue?

cheers, paul 

[1] http://sourceforge.net/projects/queue/
[1] http://packages.debian.org/stable/admin/dqs
[2] http://www.openpbs.org/
[3] http://distribulator.sourceforge.net/


-- 
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org



----- End forwarded message -----