Re: Adiscon LogAnalyzer? rsyslog + mongodb?

[Daniel Pocock <daniel@pocock.pro>]

On 04/03/14 18:04, Nicolas Dandrimont wrote:
> * Daniel Pocock <daniel@pocock.pro> [2014-03-04 15:49:25 +0100]:
>
>> I didn't see any existing package of LogAnalyzer from Adiscon, the
>> people who make rsyslog - is there any specific reason for not packaging
>> it or it is just not something anybody needed yet?  It is GPL:
>>
>> http://loganalyzer.adiscon.com/
>>
>> http://download.adiscon.com/loganalyzer/loganalyzer-3.6.5.tar.gz
>>
>> The rsyslog mongodb output module and the PHP mongodb modules are now in
>> wheezy-backports.  This would appear to be sufficient to do something like:
>>
>>     rsyslog => mongodb => loganalyzer
>>
>> Has anybody else tried that or does anybody have any comments on it (or
>> recommended alternatives)?
>>
>> http://loganalyzer.adiscon.com/articles/using-mongodb-with-rsyslog-and-loganalyzer/
> Hi,
>
> At work, I have been investigating the ElasticSearch + Logstash[1] + Kibana[2]
> combo, which has been pretty solid in my tests so far (feeding it 10GB or so of
> firewall logs a day, yes, that thing is noisy).
>
> There is no Debian packaging of that stack yet (the RFP of logstash is at [3]),
> and I haven't investigated the upstream-provided repositories either (AIUI,
> they appeared after my tests, so I ran the stuff from the "flatjar" bundle, ick).

This is obviously the more advanced and feature-rich solution (and I
notice they include syslog in their long list of connectivity options):

http://cookbook.logstash.net/recipes/rsyslog-agent/
  "The logstash agent, when run from java, can incur significant
overhead. The minimum memory footprint I have been able to achieve is
about 100mb. On tiny virtual machines, this may not be acceptable, so
you need an alternative."  (an alternative that every Debian system
contains already being a good choice)

I had a look at the distribution artifact (the flat JAR), it is a mix of
Java and Ruby, including various dependencies.  As everything is merged
into a single JAR it wasn't immediately obvious what the dependencies
are and which ones are essential but I'm guessing from the long list of
connector options that some are optional and packaging may involve
allowing people to mix and match.

For my own use cases, a Debian package isn't always a requirement and
the features this offers appear more compelling.

For some people (and some of my own use cases), LogAnalyzer is probably
enough as well and if a trivial integration with mongodb is going to be
possible in jessie, I felt it would be a nice thing for Debian.