Bug#924401: base-files fails postinst when base-passwd is unpacked
Hi!
On Tue, 2019-03-12 at 16:17:10 +0100, Helmut Grohne wrote:
> Package: base-passwd,base-files,debian-policy
>
> Debian policy section 3.8 says:
> | Essential is defined as the minimal set of functionality that must be
> | available and usable on the system at all times, even when packages
> | are in the “Unpacked” state.
The behavior for Essential package is scattered around the policy
document, there's also this in § 6.5 for the preinst maintscript:
,---
The package will not yet be unpacked, so the "preinst" script
cannot rely on any files included in its package. Only essential
packages and pre-dependencies ("Pre-Depends") may be assumed to be
available. Pre-dependencies will have been configured at least
once, but at the time the "preinst" is called they may only be in
an “Unpacked” or “Half-Configured” state if a previous version of
the pre-dependency was completely configured and has not been
removed since then.
`---
I do think the same things that currently apply to Pre-Depends,
apply to the pseudo-essential set here, even if this is probably
a bit implicit.
> Now we can make a choice:
On Tue, 2019-03-12 at 16:32:30 +0000, Simon McVittie wrote:
> On Tue, 12 Mar 2019 at 16:17:10 +0100, Helmut Grohne wrote:
> > A. /etc/passwd is part of base-passwd's interface and base-files is
> > right in relying on it working at all times. Then base-passwd is rc
> > buggy for violating a policy must. Fixing this violation is
> > technically impossible.
(This assumes that policy covers the bootstrapping process, which I
don't think it has ever done.)
> If it isn't implementable then it's probably the wrong design.
This is not possible right now, but I think it should be possible in
the future.
> Strictly speaking, I think /etc/passwd *is* part (or maybe all?) of
> base-passwd's Essential interface, but then the Policy requirement that
> it provide this interface even when unconfigured is unimplementable, and
> we can't do unimplementable things.
Yes and no (see above). I also agree these are part of the Essential
interface, but I do not agree it's specified when it is unconfigured
*and* has never been configured before.
> I think it would be reasonable to say that Essential packages are *not*
> entitled to assume that base-passwd provides /etc/passwd, even though
> non-Essential packages are entitled to assume it.
Taking this in its general form seems extremely restrictive to me,
when the only problematic case here is for when the Essential packages
have never been configured before. This would mean possibly more
painful or no logic at all for upgrades and similar.
> > B. /etc/passwd is not part of base-passwd's interface and base-files
> > wrongly relies on its presence rendering base-files rc buggy.
(I think this is also the wrong interpretation, and would mean no
other package could rely on these being always present.)
> Perhaps base-files should use chown 0:0, etc.? That would be more robust.
While that would indeed solve part of the problem, and it's actually
what dpkg is doing itself:
<https://git.dpkg.org/cgit/dpkg/dpkg.git/tree/debian/dpkg.postinst#n25>
it's still a workaround to the more general problem presented here.
> > Given that
> > we have debootstrap, cdebootstrap, multistrap, and mmdebstrap, it
> > seems like specifying the bootstrap interface would be a good idea.
> > Unfortunately, I don't exactly understand the bootstrap interface at
> > present. In practise, you cannot run postinsts of essential packages
> > in arbitrary order.
>
> This is certainly more fragile than I'd hope: I've seen debootstrap fail
> in Open Build Service chroots when presented with a modified Essential
> set (in a Debian derivative targeting containers that are not multi-user
> systems and never run on bare metal, which doesn't need everything that a
> "real" Debian system does).
>
> If we rely on bootstrap implementations having out-of-band knowledge
> of the right order to configure the Essential set, the risk is that
> they need to have different out-of-band knowledge for different target
> distributions, leading to the bootstrap implementation becoming relatively
> tightly coupled to the target distribution.
Yes, off-loading this knowledge from the packages themselves into
external bootstrapping tools is bogus IMO, and something we should
try to fix.
> Maybe the rule should be to retry configuration of each unconfigured
> package until either they all succeed, or forward progress stops being
> made? Pythonesque pseudocode:
I think this would too be a non-ideal workaround.
On Tue, 2019-03-12 at 16:17:10 +0100, Helmut Grohne wrote:
> C. Guillem Jover hinted that policy expects every essential package to
> be configured at least once. The current text does not make this
> assumption clear. If it holds, policy would simply say nothing about
> how to bootstrap an essential system, which may be fine. Given that
> we have debootstrap, cdebootstrap, multistrap, and mmdebstrap, it
> seems like specifying the bootstrap interface would be a good idea.
> Unfortunately, I don't exactly understand the bootstrap interface at
> present. In practise, you cannot run postinsts of essential packages
> in arbitrary order.
I don't think this represents my position on this issue. What I tried
to mean with this, is that the bootstrapping process is not (currently)
under the policy realm, so it's not fair to use policy to catalogue
such issues as policy-buggy.
My position is that our bootstrapping process is suboptimal, and we
should strive to improve it, in a general way, and document that. My
original thinking can be found at:
<https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap>
Thinking about this some more, and reanalyzing the current set of
bootstrap code used during pseudo-Essential package preinst/postinst,
I think there's actually another perhaps nicer option than having to
specify the entire external environment from where the bootstrapping
program is going to run (as stated in the above spec).
But first, let's see what the pseudo-Essential packages require
currently for the bootstrapping case ("install"/"configure" w/o a
version argument, ignoring debconf usage):
$ dpkg-query -f '${binary:Package}\n' -W \
$(/usr/sbin/debootstrap --variant=minbase --print-debs \
--exclude=apt,e2fsprogs sid /tmp/sid-tmp 2>/dev/null) \
| grep -v :i386 \
| while read pkg; do
ls -1 /var/lib/dpkg/info/$pkg.*inst 2>/dev/null; \
done
* /var/lib/dpkg/info/base-files.postinst
- Creates directories with proper owner/permissions.
- Creates symlinks for the default dpkg origin file.
- Creates compat symlinks for some directories.
- Initializes config files from templates.
- Initializes state files from templates.
- Initializes "log" files.
- Initializes /run file. This should be removed, as /run is
supposed to be clean after each boot.
- Initializes various dpkg files. This should probably be removed.
* /var/lib/dpkg/info/base-passwd.postinst
- Ininitializes config files from templates.
- Runs update-passwd, although I think this should not need to be
run on the bootstrap case.
* /var/lib/dpkg/info/base-passwd.preinst
- Ininitializes config files from embedded templates in the
maintscript. Duplicates the logic in the postinst script.
* /var/lib/dpkg/info/bash.postinst
- Installs alternatives.
- Adds itself into /etc/shells via add-shell.
* /var/lib/dpkg/info/bash.preinst (compiled binary)
- Handles the /bin/sh symlink creation if missing, and diversion
installation. But perhaps this can be removed now, ISTR a bug
report requesting just that, but cannot find it.
* /var/lib/dpkg/info/coreutils.postinst
- Creates /usr/bin/ to /bin/ transition symlink.
[ Ddue to merged-usr-via-symlinks. :( … ]
* /var/lib/dpkg/info/dash.postinst
- Handles /bin/sh symlink creation if missing, and diversion
installation.
- Adds itself into /etc/shells via add-shell.
* /var/lib/dpkg/info/debconf.postinst
- Removes an obsolete conffile.
* /var/lib/dpkg/info/debconf.preinst
- Removes an obsolete conffile.
- Renames a conffile.
* /var/lib/dpkg/info/debianutils.postinst
- Initializes /etc/shells from a template.
- Creates /usr/bin/ to /bin/ transition symlink.
[ Due to merged-usr-via-symlinks. :( … ]
* /var/lib/dpkg/info/dpkg.postinst
- Initializes its database and log files with owner and
permissions. This should probably be done at run-time instead,
there's already a report about this.
* /var/lib/dpkg/info/libc-bin.postinst
- Initialize config file from template.
- Runs ldconfig.
* /var/lib/dpkg/info/libc6:amd64.postinst
- Removes hwcappkgs registry. Only relevant on upgrades.
- Handles nohwcap file removal. Only relevant on upgrades.
- Restarts init. Only relevant on ugprades
* /var/lib/dpkg/info/libc6:amd64.preinst
- LD_LIBRARY_PATH sanity checks. Only relevant on upgrades.
- Kernel version requirement checks.
* /var/lib/dpkg/info/libgcc1:amd64.postinst
- Handles directory to symlink transition.
Only relevant on upgrades.
* /var/lib/dpkg/info/libpam-modules:amd64.postinst
- Creates empty config files.
* /var/lib/dpkg/info/libpam-modules:amd64.preinst
- Does nothing during bootstrapping.
* /var/lib/dpkg/info/libpam-runtime.postinst
- Run pam-auth-update.
- Initialize config file from template.
* /var/lib/dpkg/info/libpam0g:amd64.postinst
- Does nothing during bootstrapping.
* /var/lib/dpkg/info/login.postinst
- Obsolete init script conffile removal.
- Create config files with owner and permissions.
* /var/lib/dpkg/info/login.preinst
- Does nothing during bootstrapping.
* /var/lib/dpkg/info/mawk.postinst
- Installs alternatives.
* /var/lib/dpkg/info/passwd.postinst
- Fixes permissions for log files, should only be needed on
upgrade, and it should actually be removed now.
- Handles shadow group addition, only relevant for upgrades.
- Runs shadownconfig.
- Runs systemd-tmpfiles.
* /var/lib/dpkg/info/passwd.preinst
- Does nothing during bootstrapping.
* /var/lib/dpkg/info/tar.postinst
- Installs alternatives.
* /var/lib/dpkg/info/tzdata.postinst
- Does nothing during bootstrapping. The actual initial timezone
setup is off-loaded to the bootstrapping programs. :/
* /var/lib/dpkg/info/util-linux.postinst
- Installs alternatives.
- Removes obsolete alternatives.
- Runs helpers for sysvinit/systemd setup.
So from the above, I actually think now, the better solution is
probably to try to make the bootstrap case require 0 code execution
from maintainer scripts. So the requirement would be for any
pseudo-Essential package to have something along the lines of:
if [ "$1" = configure ] && [ -z "$2" ]; then
exit 0
fi
For the cases above, dpkg could handle most of these via:
- Declarative alternatives.
- Declarative diversions.
- Declarative file matadata.
- With support for ghost files.
- With support for files initialized from a template.
- Declarative config files / conffiles.
- With support for files initialized from a template.
The rest could be done by fixing other code, f.ex.:
- Add support for /etc/shells.d/ fragment files, to avoid needing
add-shell.
- Fixing programs to generate required, but missing files.
So then the bootstrapping process would pretty much require unpacking
everthing and then configuring via dpkg (either from outside or
inside the chroot), which avoids having to define the complete outter
environment. :)
Thanks,
Guillem
Reply to: