Re: Journald's qualities
>> Oh, that bug report is quite interesting, thanks.
>> Makes one wonder why they don't use naive append-only "plain text"
>> logs (tho with appropriate delimiters (maybe some kind of CSV) to make
>> searches more reliable than with old-style plain text logs)?
>>
>> What are the advantages of journald's representation?
>> I mean, to justify the slow search and large disk space usage, there
>> is presumably some upside for some use cases.
>
> The advantage of binary format is that you have separate fields for
> program, level etc and it is easier to filter by exactly what you want.
> So "systemctl status servicename" just asks log subsystem for exactly
> what it needs.
That's why I said "with appropriate delimiters (maybe some kind of
CSV)". Really plain text is what we had before, but it's not rocket
science to have a "plain text" format with a reliably delimited set
of fields, which is what I'd assume as the obvious naive choice.
>> argument against Sqlite based on the size of Sqlite,
> I'd argue that's not really technically relevant
I did say "weak" 🙂
>> but what are the advantages of journald's representation compared to
>> a naive one?
> in short: querability without text parsing. That's about it.
They have to parse the binary format, so that's not in and of itself an
upside compared to parsing CSV.
I've made my share of bad design decisions that don't pan out.
But there's always an upside to my decision (even when it turns out it
speeds up only those cases which can never occur, because of some other
aspect of the system).
AFAICT the format is *not* just a plain sequence of log entries, so
there's some additional structure which is intended to speed up
some operations.
IOW, even if contrived, there should be *some* use case where it does
better than CSV, no?
> It does have some features around FSS (signing logs) so for folks that
> need it there isn't many alternatives as it would be hard to make
> similar feature on database.
That should be just as easy to add to a CSV-style representation, tho.
> But on flipside inability to separate log into log rotation groups is
> *nasty*. You basically can't have verbose service in the system as that
> will rotate away any old logs of other services away. And looking for
> those logs is also the worst case:
It shouldn't be difficult to add a filtering step to the log rotation
(just extract all the log entries, filter out the unwanted ones, and
create a new log file with the result).
Beside the obvious usual issues of atomicity, the only real problem
I can imagine there is FSS for which you'd need to seal separately the
groups that share differently filtering rules.
> I guess another one being appendable log is that in case of crash it's
> easier to recover than embedded database...
AFAICT, CSV shares this property just as well.
> One-service-per-file approach is honestly good enough for most stuff.
> PITA to get chronological order though, every approach really have
> some drawbacks and benefits.
If you use a reasonable format with precise enough timestamps, you
should be able to "weave" entries from separate files back into
a coherent single chronology (assuming all those entries got their
timestamps from the same journald in the first place).
Stefan
Reply to: