Re: OT: launching jobs in a combined serial parallel way
On Fri, 26 Jun 2009 18:52:47 +0300
Micha Feigin <michf@post.tau.ac.il> wrote:
> On Wed, 24 Jun 2009 21:28:53 -0400
> Johan Kullstam <kullstj-ml@comcast.net> wrote:
>
> > Kamaraju S Kusumanchi <raju.mailinglists@gmail.com> writes:
> >
> > > I have three programs - say proga, progb, progc.
> > >
> > > proga, progb are completely independent. They take couple of hours to
> > > finish. The time to complete proga, progb are not same.
> > >
> > > progc should to be launched only after both proga, progb are finished. progc
> > > takes another couple of hours to finish.
> > >
> > > What is good way to automate this problem (that is no manual interaction)? I
> > > prefer to use nohup since sometimes I have to log out of the machine before
> > > the whole process finishes.
> > >
> > > Currently I have a shell script that works as below.
> > > 1) launch proga, progb in the background using nohup.
> > > 2) Ask proga, progb to write a file when they finish.
> > > 3) Every five minutes check if these files are present. If they are present,
> > > launch progc.
> >
> > Use make. I like to use make for all sorts of things, not just
> > compiles. And, since you are already using the existance of a file to
> > check for done, you are practically there. Make with the "-j" parameter
> > will spawn parallel jobs for you.
> >
> > Write a makefile similar this:
> >
>
> I think that you also need to create the files, something like
>
> >
> > proga.out:
> > proga
> touch proga.out
> >
> > progb.out:
> > progb
> touch progb.out
> >
This may work even better
> > probc.out: proga.out progb.out
> > progc
rm -f proga.out
rm -f progb.out
> >
> >
> > Then you can do
> >
> > $ nohup make -j 2 &
> >
> > Maybe a redirect of the text output to a file. Now you can walk away.
> >
> >
>
>
Reply to: