[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#727708: upstart and upgrading from sysvinit scripts



On Sun, Dec 29, 2013 at 11:21:07AM +0100, Josselin Mouette wrote:
> Le dimanche 29 décembre 2013 à 01:10 -0800, Steve Langasek a écrit : 
> > If I'm not mistaken (no references to hand - sorry), systemd upstream has
> > claimed in the course of discussions on debian-devel that lazy activation is
> > not the purpose of socket-based activation, and that using socket-based
> > activation does not require you to pay the service startup penalty at the
> > time of first connection.  However, this is not borne out by my experiments
> > with systemd on Fedora (which I would presume to be the go-to source for
> > best practices of systemd service activation).

> > On Fedora 20, after enabling the sshd and rsync service+socket units (both
> > installed but disabled by default on Fedora per their policies on running
> > services out-of-the-box) and rebooting, I find that both port 22 and port
> > 873 are bound by pid 1.  Only upon connecting to the socket does systemd
> > actually spawn the server (in the case of sshd, it spawns it as 'sshd
> > -i', so has to start up the server anew on each connection; in the case of
> > rsyncd, the .service definition is completely incompatible with socket-based
> > activation and any attempt to connect results in the rsyncd.socket unit
> > landing in a 'failed' state).

> I’m not sure you can conclude that socket activation is broken from such
> investigations. It looks to me that: 
>       * Fedora deliberately used an inetd-like sshd setup, which is more
>         suitable for a workstation to which you don’t ssh much, but not
>         for a production server. 
>       * You found a bug in Fedora’s rsyncd unit files.

> If you don’t want lazy activation, you just need to add a
> WantedBy=multi-user.target. This way, sockets will be bound by systemd
> at the earliest possible time, and passed to the daemon as it is
> started, but it will be started regardless of an incoming connection.

> This is described in more detailed in the “systemd for administrators”
> series:
> http://0pointer.de/blog/projects/socket-activation2.html

It's quite possible that I am doing something wrong, but I don't think this
is it.  Each of the .service units in question already had
'WantedBy=multi-user.target', and each of the .socket units had
'WantedBy=sockets.target'; on Fedora these were all disabled by default (to
avoid any open ports by default), but upon enabling both the service and
socket units, I get the behavior described.

I was seeing the same behavior with the lbcd package in Debian, but it turns
out this is due to the 'mutli-user' typo in lbcd.service.  Once I've fixed
this (and created the proper 'enabled' symlink), I do see the lbcd process
being started at boot.  So that much does seem to work as described, on
Debian.  I'm not sure what to make of the Fedora setup, then, because other
services that are linked into /etc/systemd/system/multi-user.target.wants do
start up at boot, but neither sshd nor rsyncd is started when the .socket is
enabled.  In that case, my concern is a different one - how can anyone claim
that systemd's socket activation is ready for prime time if even a service
as important as sshd hasn't been debugged in Fedora, one of the flagship
adopters of systemd?  (BTW, there's also both an sshd.service and an
sshd@.service here, adding to the confusion.  I've attached all of the sshd
units in case you want to look at them.)

This still leaves the concern I have about start-time races.  According to
systemd.unit(5), using 'Requires=', as Uoti suggested to Russ, does *not*
guarantee ordering:

  Note that requirement dependencies do not influence the order in which
  services are started or stopped.  This has to be configured independently
  with the After= or Before= options.  If a unit foo.service requires a unit
  bar.service as configured with Requires= and no ordering is configured
  with After= or Before=, then both units will be started simultaneously and
  without any delay between them if foo.service is activated.

In my earlier investigations (which were on Fedora 17, IIRC), I definitely
found races where a service configured with a corresponding .socket would
*sometimes* start at boot time before the socket is bound, causing it to
fall back to its own config file and binding to its own sockets... which
could result in a completely different set of sockets being bound, and
potentially introducing an unexpected security hole if the admin isn't
diligently keeping the two implementations in sync.  Since
LISTEN_FDS/LISTEN_PID is the defined API for systemd passing the socket
information to the service, for systemd to ever fail to pass this socket
information, resulting in the service deciding that it's not *actually*
running under systemd and should fall back to a different mode, is
potentially a very serious problem.

Of course, it's possible that this has been fixed in systemd since the last
time I looked.  I'll try to set up a reproducible test case for
consideration.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org
[Unit]
Description=OpenSSH server daemon
After=syslog.target network.target auditd.service

[Service]
EnvironmentFile=/etc/sysconfig/sshd
ExecStartPre=/usr/sbin/sshd-keygen
ExecStart=/usr/sbin/sshd -D $OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartSec=42s

[Install]
WantedBy=multi-user.target
[Unit]
Description=OpenSSH per-connection server daemon
Wants=sshd-keygen.service
After=auditd.service sshd-keygen.service

[Service]
EnvironmentFile=-/etc/sysconfig/sshd
ExecStart=-/usr/sbin/sshd -i $OPTIONS
StandardInput=socket
[Unit]
Description=OpenSSH Server Socket
Conflicts=sshd.service

[Socket]
ListenStream=22
Accept=yes

[Install]
WantedBy=sockets.target

Attachment: signature.asc
Description: Digital signature


Reply to: