Re: Second take at DEP17 - consensus call on /usr-merge matters
On Sun, 9 Jul 2023 at 22:07, Helmut Grohne <helmut@subdivi.de> wrote:
>
> Hi Sam,
>
> Thanks for trying to wrap your head around the complexity.
>
> On Sat, Jul 08, 2023 at 07:57:40AM -0600, Sam Hartman wrote:
> > So for me, a 3C proposal would have two components:
> >
> > 1) An explanation of what the archive looks like at time of bootstrap
> > (and changes to any bootstrap programs) so I can reason about whether
> > bootstrap works.
>
> I hope this one is simple.
> * All packages ship all of their files in canonical locations.
> * base-files ships all of the aliasing symbolic links and their target
> directories.
> * Given that base-files installs the symbolic links, all programs are
> immediately working after unpack prior to running any maintainer
> scripts.
> * Consequently, cdebootstrap and mmdebstrap just work without any
> modification.
> * An unmodified debootstrap fails (unpacking base-files due to -k).
> + We modify debootstrap such that it first unpacks and then merges.
> OR
> + We stop passing the -k flag to tar. (Though we need a better
> understanding for why that was added post jessie.)
>
> > 2) An argument of safety of upgrades focused on the changes and why
> > those changes are safe both for unstable upgrades and for bookworm
> > upgrades.
>
> As far as I understand the question here, it is about those aspects that
> are specific to the 3C solution as opposed to a 3B solution. In both
> cases, we move all of the files to their canonical locations. I'm not
> sure whether the protective diversions for aliasing links (DEP17-M4) are
> something we ultimately need in all scenarios, but in case of 3C, we
> quite certainly need them to make upgrades safe, so that's an aspect to
> consider here. The other aspect of course is shipping the symlinks in
> base-files (DEP17-M11). So what could go wrong here?
>
> In an upgrade scenario from unstable or from bookworm, we'd have to
> unpack and configure usrmerge-support before unpacking base-files, since
> that becomes a Pre-Depends of base-files. usrmerge-support.preinst would
> verify that the filesystem is merged already (much like usr-is-merged
> does) except that it does not tolerate
> /etc/unsupported-skip-usrmerge-conversion anymore, so any system using
> that mechanism will fail this preinst. Then usrmerge-support.preinst
> would install the protective diversions (DEP17-M4) on behalf of
> base-files. Since these are --no-rename, the filesystem is not modified
> and since we just verified that all the affected locations really are
> symbolic links rather than directories, dpkg-divert wouldn't error out
> about diverting a directory. In any case, usrmerge-support is eventually
> configured (without a postinst), which allows unpacking base-files.
> Whenever we unpack base-files (now or for subsequent upgrades), dpkg
> will create each aliasing symlink with a temporary name and rename(2) it
> to the final destination. Since rename(2) atomic, the aliasing symlinks
> will be never go missing. When upgrading or removing any other package,
> dpkg may consider removing an aliasing symlink as that package may be
> the last package to ship an aliased file. When this happens, the removal
> of the directory will be redirected to an innocent location via the
> protective diversion. Since diversions only match exactly (they are not
> meant to be used for directories), files installed "below" the diverted
> aliasing links (i.e. aliased files) will be entirely unaffected by the
> protective diversions and dpkg will operate on them as usual.
>
> The most common failure mode during upgrades seen by users likely will
> be when /etc/unsupported-skip-usrmerge-conversion exists and the system
> isn't actually merged.
>
> I have a hard time figuring out what else could go wrong here and that's
> probably because I'm biased towards 3C. On the other hand, the reason
> for me to like it is because I see very little that could go wrong (in
> addition to what can already go wrong due to moving all the files). I
> hope that others can use this detailed description of what happens to
> construct possible failure cases such that we can better understand the
> risk here.
You have said in the original mail and on the writeup that this option
requires all the affected packages to be upgraded at the same time,
and in the correct order, or things will break. What happens if any of
those packages are held back, for whatever reason? This is the
fragility aspect that I am worried about, and that is not an issue at
all if we just fix mmdebstrap to do the right thing as debootstrap
already does.
Kind regards,
Luca Boccassi
Reply to: