Tool for immediate tabulation of [shorewall] ulog files

["Jeffrey B. Green" <jeff@kikisoso.org>]

Hi,

I didn't find anything for immediate use to tabulate occurrences in
shorewall ulog files and so I wrote the below script after doing
similarly on the command line. I needed it to focus in on behavior that
showed up at a higher level. I offer it below for general use via GPL. I
believe there are no present bugs, however I keep polishing (revising)
it and don't do any systematic regression testing and so can offer no
guarantees, nor any particular coding standard.

-jeff

#!/bin/ksh

#
#	Author: Jeff Green (2-1-09)
#	nb: This cmd requires the input to be in ulog format
#	License: GPLv3 or any later GPL license.
#

prog=`basename $0`
usage()
{
    echo "Usage: [ zcat zipped_ulog_files | ] cat ulog_files [-] | $prog [-utsdnSDOh] pattern"
}

help()
{
    echo -e "\
\t-u\trestricted to UDP messages	\n\
\t-t\trestricted to TCP messages	\n\
\t-s\ttablulate source IP addresses	\n\
\t-d\ttablulate destination IP addresses	\n\
\t-S\ttablulate source PORT numbers	\n\
\t-D\ttablulate destination PORT numbers	\n\
\t-n\tdo not output day tabulation table	\n\
\t-O\toutput a sorted (Ordered) by count table	\n\
\t-h\tThis message";
}

unset UDP TCP FKEY ENUM NODATE PORT PKEY
argcnt=0
while getopts utsdnSDOh opt ; do
        case "$opt" in
        u) UDP='| grep "PROTO=UDP" ' ; argcnt=$((argcnt+1)) ;;
        t) TCP='| grep "PROTO=TCP" ' ; argcnt=$((argcnt+1)) ;;
        s) ENUM=1 ; FKEY=9 ; argcnt=$((argcnt+1)) ;;
        d) ENUM=1 ; FKEY=10 ; argcnt=$((argcnt+1)) ;;
        n) NODATE=1 ; argcnt=$((argcnt+1)) ;;
        S) ENUM=1 ; PORT=1 ; PKEY=1 ; argcnt=$((argcnt+1)) ;;
        D) ENUM=1 ; PORT=1 ; PKEY=2 ; argcnt=$((argcnt+1)) ;;
        O) ORDERED='sort -n -t":" -k2' ; argcnt=$((argcnt+1)) ;;
        h) usage; help; exit 0 ;;
        *) usage; exit 1 ;;
        esac
done

if
  [  ! -z "$UDP"  -a  ! -z "$TCP" ]
then
  echo "$prog: both -u and -t cannot be set"
  exit 1
fi

if
  [  ! -z "$ORDERED"  -a  -z "$FKEY" -a -z "$PKEY" ]
then
  echo "$prog: -O option is irrelevant w/o the -s, -d, -S, or -D option"
  exit 1
fi

shift $argcnt

if
  [ $# -ne 1 ]
then
  usage
  exit 1
fi

unset CNT CIP CPORT
[ -z "$NODATE" ] && typeset -A CNT
[ ! -z "$ENUM" ] && typeset -A CIP
[ ! -z "$ENUM" -a  ! -z "$PORT" ] && typeset -A CPORT
ITER=0
CMD="grep \"$1\" ${UDP:-} ${TCP:-}"

cat - | sh -c "$CMD" | while read line
do
if
  [ -z "$NODATE" ]
then
  DATE=`echo $line | cut -d' ' -f1-2 | tr " " "_"`
  CNT["$DATE"]=$((CNT["$DATE"] + 1))
fi
if
  [ ! -z "$ENUM" -a ! -z "$FKEY" ]
then
  DST=`echo $line | cut -d' ' -f${FKEY} | cut -d'=' -f2`
  CIP[$DST]=$((CIP[$DST]+1))
fi
if
  [ ! -z "$ENUM" -a ! -z "$PORT" ]
then
  PT=`echo $line | sed -e 's/^.*SPT=/SPT=/' | cut -d' ' -f${PKEY} | cut -d'=' -f2`
  CPORT[$PT]=$((CPORT[$PT]+1))
fi
done

if
  [ -z "$NODATE" ]
then
for i in ${!CNT[*]}
do
echo $i - ${CNT["$i"]}
done | sort -t' ' -k1
fi

if
  [ ! -z "$ENUM" -a ! -z "$FKEY" ]
then
for i in ${!CIP[*]}
do
echo "$i:${CIP[$i]}"
done | sh -c "${ORDERED:-cat -}"
fi

if
  [ ! -z "$ENUM" -a ! -z "$PORT" ]
then
for i in ${!CPORT[*]}
do
echo "$i:${CPORT[$i]}"
done | sh -c "${ORDERED:-cat -}"
fi