Re: OT: launching jobs in a combined serial parallel way

[Micha Feigin <michf@post.tau.ac.il>]

On Wed, 24 Jun 2009 21:28:53 -0400
Johan Kullstam <kullstj-ml@comcast.net> wrote:

> Kamaraju S Kusumanchi <raju.mailinglists@gmail.com> writes:
> 
> > I have three programs - say proga, progb, progc.
> >
> > proga, progb are completely independent. They take couple of hours to
> > finish. The time to complete proga, progb are not same.
> >
> > progc should to be launched only after both proga, progb are finished. progc
> > takes another couple of hours to finish.
> >
> > What is good way to automate this problem (that is no manual interaction)? I
> > prefer to use nohup since sometimes I have to log out of the machine before
> > the whole process finishes.
> >
> > Currently I have a shell script that works as below.
> > 1) launch proga, progb in the background using nohup.
> > 2) Ask proga, progb to write a file when they finish.
> > 3) Every five minutes check if these files are present. If they are present,
> > launch progc.
> 
> Use make.  I like to use make for all sorts of things, not just
> compiles.  And, since you are already using the existance of a file to
> check for done, you are practically there.  Make with the "-j" parameter
> will spawn parallel jobs for you.
> 
> Write a makefile similar this:
> 

I think that you also need to create the files, something like

> 
> proga.out:
>              proga
	touch proga.out
> 
> progb.out:
>              progb
	touch progb.out
> 
> probc.out:   proga.out progb.out
>              progc
> 
> 
> Then you can do
> 
> $ nohup make -j 2 &
> 
> Maybe a redirect of the text output to a file.  Now you can walk away.
> 
>