Re: Apache2 VHosts and AWStats within a clustered environment?

[norbert_schuetz@bigfoot.com]

Hi Matthew,

we tried both variants, NFS and copying logfiles nightly.
NFS was a performance killer and somewhat unstable then (think concurent
writes). Copying/merging/sorting several hundred GB logdata nightly ran
into problems once the volume was too large to be handled within 24
hours ("daily stats"). Having logs on an NFS host and processsing them
via NFS (from the web servers, as you suggested) at least doubles
network traffic, so this never was an option to us even then. But ymmv,
it could work for small businesses ;-)

Did you check out the log-pipe feature of Apache?
http://httpd.apache.org/docs/2.2/logs.html#piped
You might be able to distribute logging to one ore many central log
hosts this way. We use a homemade daemon on each web server to
streamline the piping somewhat (caching, some pre-processing etc.)
Additional plus: You can accumulate data into near-realtime stats for
your customers, set up traffic limits, accounting, you name it. Just
make sure you have sufficient bandwidth so you do not stuff up the lines
with log traffic. Maybe just add an extra hardware interface for logging
(and a switch, if this is the bottleneck).
Then set up as many stats-servers as needed to process the logfiles in
time and for your customers to grab their graphs.

Additional tips: Set up your own Aapche logfile format to include the
vhost name ;-) If your customers are used to find their stats on, say
theirdomain.com/stats/ just set up a system-wide redirect to your
stats-server(s). Ah, and: We do not offer on-the-fly awstats generation,
so that eases the pain somewhat ;-)


Cheers, Norbert