On Friday December 28 2007 10:23:26 Douglas A. Tutty wrote: > On Fri, Dec 28, 2007 at 06:58:42AM -0600, Ron Johnson wrote: > > On Friday December 28 2007 00:27:24 Jerome BENOIT wrote: > > [snip] > > > > > I have a long list of jobs that I want to launch on my box. > > > On a one core box, I can make a for loop in order to submit > > > then in sequence. On a multi-core box, this approach does > > > not work anymore. Hence my intention to use Torque: here I > > > envisage my multi-core box as a cluster where each core is > > > a node. Note that you can configure LAM in such a way. > > > > Old school OSs like OS/MVS, DOS/VSE, OpenVMS, MPE, MCP, etc > > all have sophisticated batch queue functionality. > > > > I've never understood why Unix has never received such power. > > Because nobody has written it. Ron, I've never run any of > those OSs. AIUI, you had queues: one for heavy IO, one for > heavy comp, one for a mix. The queue manager ran one heavy IO > with one heavy comp, but only one mixed one at a time. IIUC, > this would require the operator/programmer to classify the > program as to which queue it should reside. That's not how it works on z/OS (OS/MVS), DOS/VSE & OpenVMS. On them, you have "named batch queues". Each queue has a default (in Unix terminology) niceness level, and "width" (like how a bank branch has a single line feeding multiple teller windows, the "width" defines the number of jobs that can be in the Running state, whereas the jobs standing in "line" are Pending). Also, there is a job priority, so that important Pending jobs jump to the front of the line, and less important jobs fall to the rear. Just like with "at", jobs can be scheduled to run at various times. Of course, if the execute slots are all full at a job's run time, it goes not to Executing but to Holding state. You can have as many named batch queues as you want, but typically only a few are used. The beauty of batch queues is that they are dynamic. So, for example, at 18:00 after all the users have logged off you increase the queue width so that more batch jobs can run, and at 06:00 you shrink the queue width, so that by 08:00 the "excess" batch jobs have completed and only a few can run at a time, thus giving more resources to the on-line users. The most impressive product, though, IMNSHO, was DEC SCHEDULER (now CA Unicenter Job Management). Like cron it could schedule things by the minute thru dow & month, but also at fiscal intervals, and had pre- and post-functions and allowed for complex dependencies. Of course, since it was a DEC product that ran on VMS, it was cluster-aware and could run jobs on remote nodes thru DECnet (think NFS on steroids, or SMB if it were written not by MSFT but by a company that knew what it was doing). > Right now, with cron, jobs get done sequentially. If you watch > it with top, at any given point, you could find the system > maxing out CPU and IO being trivial, or the system spent > waiting for IO with free comp cycles available. > > It shouldn't be too hard to write program that looked at the > current load and decided which queue to select for the next > program to run. This would take into account running programes > not under controll of the queueing system (eg normal cron runs, > other users, etc). Although, IIRC, on those older OSs, the > system was either in multi-user state or batch state. Geez. When? 1965? Certainly not since the early 1970s. -- ----------------------------------------------------------------- Ron Johnson, Jr. Jefferson, LA USA "A great many open minds should be closed for repairs." Toledo Blade Newspaper
Attachment:
signature.asc
Description: This is a digitally signed message part.