Re: do packages depend on lexical order or {daily,weekly,monthly} cron jobs?
Philipp Kern writes ("Re: do packages depend on lexical order or {daily,weekly,monthly} cron jobs?"):
> On 2019-08-05 17:34, Ian Jackson wrote:
> > With current code the options are:
> >
> > A. Things run in series but with concatenated output and no individual
> > status.
> >
> > B. Things run in parallel, giving load spikes and possible concurrency
> > bugs; vs.
> >
> > I can see few people who would choose (B).
...
> > IOW reliability and proper operation is more important than separated
> > logging and status reporting.
>
> If we are in agreement that concurrency must happen with proper locking
> and not depend on accidental lineralization then identifying those
> concurrency bugs is actually a worthwhile goal in order to achieve
> reliability, is it not?
Given that the concurrency is not actually desirable, it seems foolish
to change the execution model to a concurrent one and thereby impose
a lot of additional requirements on these scripts.
Obviously I agree that bugs, where there are bugs, should be fixed.
However, I do not agree that because bugs should be fixed,
specifications should be changed to declare buggy, things which are
currently not buggy at all.
> I thought you would be the first to acknowledge
> that bugs are worth fixing rather than sweeping them under the rug.
I confess I was quite annoyed when I read this the first time. It
still irritates me.
I am not suggesting that anything should be "swept under the rug".
The existing scripts, which do not have any locking, are completely
correct. They are entitled to make the assumptions that they
currently do, because that is what the specification says and because
that is how the current callers work.
Concurrency in particular is a particularly awkward thing to retrofit,
because code which was not previously concurrent suddenly becomes
unauditably buggy, and because concurrency bugs are very hard to find,
analyse and fix.
Sometimes it is necessary to retrofit concurrency for performance
reasons. We have suffered a lot of buggy software as a result. But
in this case concurrency is simply harmful and should be avoided.
> We already identified that parallelism between the various stages
> is undesirable. With a systemd timer you can declare conflicts as
> well as a lineralization if so needed.
ISTM that a simple solution is for systemd to infer the appropriate
linearisation. I don't understand why this approach is apparently
being rejected.
> And finally, the load spikes: Upthread it was mentioned that
> RandomizedDelaySec exists. Generally this should be sufficient to even
> out such effects.
A predictable scheduler which runs all the jobs in the same order each
time, sequentially, and without gaps, is clearly better than one which
runs them at random times and hopes that the resulting schedule is a
good one.
My bottom line:
The Debian systemd maintainers are of course free to enable
parallelism for systemd users; notwithstanding that everyone
apparently acknowledges that parallelism here is undesirable.
But it would be wrong to change the Debian Policy specification for
cron.fooly script execution. There is nothing stopping systemd from
implementing the existing specification; and, there is nothing in this
specification which prevents systemd from providing whatever logging
and monitoring features are desired.
i
And there is nothing wrong with the current specification. The only
difficulty is that systemd doesn't currently implement it to the
systemd maintainers' satisfaction.
So any concurrency bugs (or other infelicities) resulting from
violations by systemd of the currently defined interface, should be
seen as bugs in that hypothetical concurrent systemd cron.fooly
arrangements, not bugs in the cron.fooly scripts provided by packages.
The solution is to improve systemd, not to change policy to divert the
blame for any bugs caused by anyway-unwanted concurrency.
Ian.
--
Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Reply to: