Re: Niced cron jobs
On Sun, Jun 27, 1999 at 12:37:04PM +1000, Craig Sanders wrote:
>
> yes, this is true - the problem is that the niced cron jobs don't get
> ENOUGH cpu to complete in a reasonable time. for some jobs - eg those
> which use up a lot of memory, like processing apache or squid log files
> or rebuilding the locate db - this can be disastrous.
The job of batch queueing systems (GNU Queue, GNQS (GPL- not packaged),
and DQS (no commercial distribution)) is to serialize resource intensive
tasks, which sounds like exactly the problem being discussed. It would be
difficult to implement, but one solution to this would be to have the
daily/weekly/monthly cron jobs check whether a queueing system was present
and submit jobs to it rather than run them directly. If not present or
configured commands could just be run as they are now. If the system is so
overloaded that jobs don't finish the queues would grow in length, but
additional processes wouldn't run until previous ones finished. Growing
queues lengths would be a clue that you were severely overloaded.
DQS at least has prioritized queues (system queue could suspend user
batch jobs, and be suspended itself when interactive activity or system load
is too high).
A qsub (posix) interface to GNU queue would probably not be difficult to
write (the DQS man pages are derived from the standard), and GNQS could be
packaged fairly easily I suspect (Linux was one of the developemnt
platforms), though development has been stalled for some time. The posix
commands that DQS currently provides could be made alternatives if multiple
posix queueing systems are packaged.
-Drake
Reply to: