RE: Simple web log analysis for multiple sites?
I'd also be interested to know what other folks are doing for this. We use
webalizer, but we keep seperate stats & reports per each web site. I then
have a program that reads the webalizer.hist file for each site and updates
an SQL table with information for each site. If someone needed more data
they could probably extract it from webalizer's HTML files, but using
webalizer.current is a bad idea since it destroys it at the end of each
month and starts over fresh. I wish it preserved that data because there is
a lot of good stuff in there I might like to use. Perhaps I'll contribute a
patch someday; but more likely we will just reinvent the wheel so we can get
over some other shortcomings of webalizer.
If this program would be useful to anyone else I could share it. Below are
some records for one site we host. The "ws" column is the web server on
which the site resides. "ds" is the datestamp column, basically in mysql
the easiest way to do this was to use the first day of each month to
represent data for the whole month. "complete" indicates if the data for
that month is either partial data, or if it has data for that month, as well
as for the following month. While that doesn't mean there are no holes in
the data, it does at least give you an indication of if you should use it
for billing/etc yet :-) ts is just a timestamp column.
- jsw
mysql> SELECT * FROM WlfMSum WHERE ws="fire" AND sn="memepool.com" AND
ds="2001-10-01"\G
*************************** 1. row ***************************
ws: fire
sn: memepool.com
ds: 2001-10-01
hits: 1133903
files: 1012502
sites: 163933
kbytes: 30512375
pages: 988517
visits: 577638
complete: 1
ts: 20011101113012
1 row in set (0.04 sec)
mysql> SELECT * FROM WlfMSum WHERE ws="fire" AND sn="memepool.com";
+------+--------------+------------+---------+---------+--------+----------+
--------+--------+----------+----------------+
| ws | sn | ds | hits | files | sites | kbytes |
pages | visits | complete | ts |
+------+--------------+------------+---------+---------+--------+----------+
--------+--------+----------+----------------+
| fire | memepool.com | 2001-08-01 | 233195 | 210099 | 47498 | 7255226 |
214404 | 122863 | 1 | 20011101113012 |
| fire | memepool.com | 2001-09-01 | 931873 | 823837 | 147449 | 29061296 |
817547 | 485257 | 1 | 20011101113012 |
| fire | memepool.com | 2001-10-01 | 1133903 | 1012502 | 163933 | 30512375 |
988517 | 577638 | 1 | 20011101113012 |
+------+--------------+------------+---------+---------+--------+----------+
--------+--------+----------+----------------+
4 rows in set (0.07 sec)
-----Original Message-----
From: John Ackermann N8UR [mailto:jra@febo.com]
Sent: Thursday, November 15, 2001 8:19 AM
To: debian-isp@lists.debian.org
Subject: Simple web log analysis for multiple sites?
Hi --
I'm looking for a program that will analyze the logs across the multiple
virtual sites that I run and provide summary-level info (e.g., number of
hits/bytes per site per day, with monthly summaries, etc).
I'm currently using a slightly hacked version of webstat with some shell
scripts that cat the various logfiles together, add an identifying tag,
sort the result, and feed it into the analyzer, but that really generates
more info than I need for top-level summary purposes and doesn't provide
easy per-site statistics.
Thanks for any suggestions.
John
jra@febo.com
----
John Ackermann N8UR jra@febo.com http://www.febo.com
President, TAPR n8ur@tapr.org http://www.tapr.org
--
To UNSUBSCRIBE, email to debian-isp-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact
listmaster@lists.debian.org
Reply to: