Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var
On Wed, 7 Jun 2023 at 11:29, Simon McVittie <smcv@debian.org> wrote:
>
> On Tue, 06 Jun 2023 at 20:40:52 -0700, Russ Allbery wrote:
> > Luca Boccassi <bluca@debian.org> writes:
> > > +Packages might need additional files or directories to implement their
> > > +functionality. Directories that are located under ``/var/`` or
> > > +``/etc/``, and files that are located under ``/var/``, must not be
> > > +created manually via maintainer scripts, but instead be declaratively
> > > +defined via the `tmpfiles.d
> > > +<https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`_
> > > +interface.
> >
> > This is an oddly specific list of directories and not at all the
> > directories that I would have expected to be handled by tmpfiles.d.
>
> Sorry, in my previous reading of this bug I had been concentrating on
> the mechanics of how to make tmpfiles.d(5) something that maintainers can
> rely on if it's convenient/helpful, and I'd missed that Luca is asking
> for its use to be mandatory in some cases.
>
> I would personally be inclined to concentrate on making tmpfiles.d(5)
> something that we can rely on and encourage the use of where appropriate,
> even on non-systemd systems, so that (upstream and downstream) maintainers
> can move towards it of their own accord because it's more convenient
> than other options, and put aside the question of making it generally a
> "should" or "must" for the moment.
>
> I believe (please correct me if I'm wrong) that Luca's intention here
> is that this is drawing a line between:
>
> - the static files of the OS: /usr and the /usr-like top-level directories
> (the ones that get merged by the /usr merge), which should be statically
> shipped in packages and managed by dpkg (or on "immutable" systems
> that use image-based/tree-based upgrades, maybe by ostree or casync
> or similar, from an tree originally constructed from dpkg packages)
>
> - this specific system's persistent state: /var and parts of /etc
>
> with the intention of eventually enabling functionality like being
> able to do a "factory reset" to the equivalent of a freshly installed
> system by deleting (most of) /etc and /var, rebooting, and letting the
> OS re-create them from a template below /usr; or doing the equivalent
> for individual packages by deleting only their part of /etc and /var.
>
> /etc is somewhere between static files and state, because traditionally
> it has been a mixture of files that the sysadmin or installer must provide
> (like /etc/passwd); configuration files that are shipped by a package and
> can be edited by the sysadmin (like /etc/systemd/logind.conf); and
> integration glue that links up one package with another, can in principle
> be edited by the sysadmin, but in practice is rarely edited
> (like /etc/profile.d/flatpak.sh).
>
> Various upstream projects including systemd have been trying to reduce
> the extent to which /etc and /var are included in the data.tar.* of a .deb
> or other packaging systems' equivalents, by moving the integration glue
> to a /usr-like directory (/lib/udev/rules.d, /usr/share/dbus-1/system.d),
> reserving the corresponding /etc directory for sysadmin configuration
> (/etc/udev/rules.d, /etc/dbus-1/system.d), and providing a way for the
> sysadmin to "mask" any integration files they want the system to ignore.
>
> If we disregard conffiles and configuration files in /etc for the
> moment, there are basically three ways for a package to get a file onto
> the running system:
>
> - ship it in the data.tar.* of a .deb
> - create it from a maintainer script and also during boot
> - maybe via tmpfiles.d(5)
> - or maybe open-coded
> - have the package create it at runtime, on-demand
> - this clearly doesn't work if the package's code runs unprivileged
> and relies on root having created a directory for it already
>
> For /usr and the /usr-like directories, shipping files in the .deb is by
> far the most common, although a few packages need to create files here
> via maintainer scripts or triggers (for example
> /usr/lib/x86_64-linux-gnu/gio/modules/giomodule.cache which is a summary
> of files created by multiple packages, and is updated whenever those
> packages are added, removed or changed).
>
> For /run, /tmp and /var/tmp, I think there's consensus that shipping files
> in those directories in the .deb is a bug, because at the next reboot,
> the file will be deleted, leaving the files that dpkg thinks it's managing
> out of sync with the files that actually exist. At the moment, these are
> variously created by maintainer scripts, systemd units/init scripts, or the
> daemons themselves, with some duplication, and no good way to get an
> overview of which packages "own" which locations: dpkg doesn't know anything
> about them, and systemd knows about some but not all of them.
>
> tmpfiles.d seems like a good way to keep track of who "owns" those
> transient files and directories. I think a Policy "must" is probably too
> strong here, but a "should" might be reasonable?
>
> For the persistent parts of /var, several packages ship regular files
> (as opposed to directories) in the .deb. I think there might be rough
> consensus that doing so is at least a "code smell", but quite a lot of
> packages do this, so a Policy "must" certainly seems too strong at this
> stage. Legacy policy files for polkitd (<< 0.106) are a notable example.
> They're no longer necessary with bookworm's polkitd, and I'm hoping to
> get rid of them during the trixie cycle; but historically /var/lib was
> the only place supported by polkitd other than /etc, so it was functionally
> necessary to ship these files.
>
> Looking at my /var, the TeX family of packages seem to be the heaviest
> users of regular files in /var, with /var/lib/tex-common/**/*.{cfg,cnf}.
>
> I think we can safely say that creating the top-level directory
> of a package's /var, /run, etc. subdirectory declaratively is more
> self-documenting than creating it imperatively, and tmpfiles.d seems
> like a perfectly good way to achieve that.
>
> I don't think it's desirable to have wording that could be taken to imply
> that packages would be wrong to create files inside those top-level
> directories at runtime - that would seem silly (for example I don't
> want anyone arguing that because the mariadb maintainer script starts
> the server, it's somehow a bug for mariadb to create arbitrary database
> tables below /var/lib/mysql).
>
> I also don't think Policy should forbid maintainer scripts using files
> inside those tmpfiles-managed directories for their own state. For example,
> maintainer scripts should be allowed to create files like
> /var/lib/mysql/debian-10.11.flag and
> /var/lib/systemd/deb-systemd-helper-enabled/*, even if we want to require
> /var/lib/mysql and /var/lib/systemd/deb-systemd-helper-enabled to be
> registered in tmpfiles.d(5) so that the sysadmin can discover where they
> come from.
Yes I agree, it was not my intention to forbid or discourage such
cases, quite the opposite. What I wanted to say is: if you are
creating directories/files by hand in a maintainer script, please use
tmpfiles.d instead, or if your service/tool/code can do it instead,
that's fine too. Among the goals that you correctly described above,
there's also the desire to remove as much custom maintainer script
code as possible, and that's what I was aiming at.
> > Packages that need to create files or directories in file systems that
> > may be deleted on each reboot (for example, ``/run``, ``/tmp``, and
> > ``/var/tmp``) should do so via configuration files in the
> > ``/usr/lib/tmpfiles.d`` directory. The syntax of those files is
> > defined by the `systemd tmpfiles.d documentation
> > <https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`__.
> >
> > However, reading that documentation, it sounds like most of the cases for
> > other directories are handled by other systemd unit configuration
> > directives. We should say that explicitly here and reproduce the list of
> > other directories that should be handled directly by the unit file if
> > that's what we want people to do.
>
> Hmm, yes. Which of these two policies do the systemd maintainers want?
>
> - If a service has RuntimeDirectory= etc. in its unit, then redundantly
> registering those directories in tmpfiles.d(5) is not required unless
> there is some technical reason to do so
>
> - If a service has RuntimeDirectory= etc. in its unit, then it must
> redundantly register those same directories with tmpfiles.d(5)
>
> Reasons we might want the first of those: "don't repeat yourself", and
> letting systemd create the directories as late as possible before starting
> the service, and clean them up as early as possible after stopping it.
>
> Reasons we might want the second: if we had the first policy, a sysadmin
> wanting to find out who "owns" /var/lib/mystery will need to check both
> tmpfiles.d and systemd units; if we had the second, in principle they only
> need to check tmpfiles.d. Also, non-systemd init systems won't read
> RuntimeDirectory= etc., so if the directory is functionally required for
> an LSB init script, it needs to be created some other way, and tmpfiles.d
> is a reasonable choice for that other way.
I would tend toward the first one, for the DynamicUser case - for
clarity, this currently means RuntimeDir and friends are recursively
chowned on the fly. I hope for Trixie we'll get uid mapping done
instead and it will be moot, but it's not implemented yet so trying to
be careful.
Also, note that as mentioned earlier these settings work well when
there is a clear and obvious owner, to whose lifecycle the directory
can be tied to, but there are cases where there's either no service at
all or no clear owner.
Reply to: