Re: Resource consumption.

[Chris Foote <chris@inetd.com.au>]

On Wed, 29 Oct 2003, Rudi Starcevic wrote:

> >>I'm pretty sure I have a cron job analysing apache logs which is
> >>consuming too much of the system's resources.
> >>So much is spent on Webalizer and Awstats that the web server stops
> >>answering requests.
> >>
> >CPU time or IO bandwidth?
>
> CPU time is what I meant. Sorry I should be more clear
>
> >>The output of `uptime` was something like 2.2 before I manually kill the
> >>script and all is OK again.
> >
> >2.2 should not be a great problem.  A machine that has a single CPU and a
> >single hard disk probably won't be giving good performance when it's load
> >average exceeds 2.0, but it should still work.
> >
> I thought that is the load average went about 1.0 that this was bad and
> mean you need
> to do something to help bring the load under 1.0.
>
> Even one process of Awstats uses heaps of CPU - over 90%.
> Maybe I need to create a user account for processing Apache logs and limit
> CPU consumption with 'ulimit' or something ??

I think you might be overlooking the value of the 'nice' shell
builtin - try:

# do webalizer
for i in /var/log/apache/access_tmp/*-access_log; do
   site=`echo $i | sed 's/\/var\/log\/apache\/access_tmp\///'`
   site=`echo $site | sed 's/-access_log//'`
   if [ -e /etc/webalizer/$site.webalizer.conf ];
    then
        nice webalizer -D /var/log/webazolver/dns_cache.db -c \
                       /etc/webalizer/$site.webalizer.conf;
    WEB_PID=$!;
    wait $WEB_PID;
    fi
done


Cheers,
Chris

	Linux.Conf.Au Adelaide Jan 12-17 2004
	Australia's Premier Linux Conference
	http://lca2004.linux.org.au