Bug#994388: tech-ctte: More specific advice regarding merged-/usr and implications of #978636
On Wed, 15 Sep 2021 at 12:35:38 +0100, Simon McVittie wrote:
> https://salsa.debian.org/debian/tech-ctte/-/blob/master/994388_merged_usr_advice/draft.md
I have updated this draft to incorporate my edits from !3, and feedback
from bremner on IRC.
I'd like to keep this moving, because it's somewhat time-sensitive: people
are already taking action based on our earlier resolution, some of which is
not necessarily the action we would have wanted them to take.
On Wed, 15 Sep 2021 at 11:46:02 +0100, Simon McVittie wrote:
> Some questions that I think we need to answer explicitly:
>
> - What do we mean by merged-/usr? (We already said this in #914897, but
> I think it's worth reiterating that symlink farms are not it.)
This is defined (again) by
https://salsa.debian.org/debian/tech-ctte/-/blob/master/994388_merged_usr_advice/draft.md#definitions-and-current-status
> - Is it OK for packages in testing/unstable during Debian 12 development
> to assume/require a merged-/usr system? (tl;dr: some maintainers think
> the answer is yes, but I think the answer is no.)
>
> - When will it be OK for packages in testing/unstable to assume/require
> a merged-/usr system? (tl;dr: some maintainers think the answer is
> "right now", but I think the answer is "when the Debian 13 cycle begins"
> and not before.)
Both of these are addressed by
https://salsa.debian.org/debian/tech-ctte/-/blob/master/994388_merged_usr_advice/draft.md#upgrade-path-from-debian-11-to-debian-12
> - Should maintainers proactively move files in packages from the root
> filesystem into /usr? (tl;dr: I think they should not, at least until
> the implications are better-understood.)
>
> - Should maintainers of tools, e.g. debootstrap, proactively move files
> in packages from the root filesystem into /usr, e.g. systemd system units?
> (tl;dr: I think they should not.)
Both of these are addressed by
https://salsa.debian.org/debian/tech-ctte/-/blob/master/994388_merged_usr_advice/draft.md#moratorium-on-moving-files-logical-locations-into-usr
> - Are packages built on a merged-/usr system required to work correctly
> on a non-merged-/usr system? If they are not, is it RC?
> (I think they are required to work, and it should usually be RC if
> they don't)
https://salsa.debian.org/debian/tech-ctte/-/blob/master/994388_merged_usr_advice/draft.md#building-packages
On Thu, 16 Sep 2021 at 15:35:11 -0700, Sean Whitton wrote:
> You write:
>
> >> - Because Debian 11 installations with the non-merged-/usr layout already
> >> exist, all packages in Debian 12 should be installable onto a non-merged-/usr
> >> system along with their dependencies, and work correctly on the resulting
> >> system.
>
> (1) The reason for this, to put it a bit simplistically, is that we
> don't require apt to perform the upgrade between stable releases in any
> particular order, right? Or are there other reasons distinct from this
> one that I'm missing? I think it would be good to state the thing about
> apt (in better language than mine) in the text.
I think that's the main reason. We have not traditionally mandated
the use of a special upgrade tool like Ubuntu's do-release-upgrade(8),
so the upgrade happens in whatever order apt chooses, which can vary
between machines.
Another reason why I think we want Debian 12 packages to be installable
onto non-merged-/usr systems is that to be able to do our development work,
they need to be installable onto testing/unstable systems, which (again)
means that the upgrade order is undefined.
We also have best-effort support for partial upgrades, either oldstable
-> stable or stable -> testing/unstable: upgrading only a subset of the
available packages, plus their mandatory dependencies, but without also
upgrading apparently-unrelated packages. This can only ever be partially
supported, because it leads to a combinatorial explosion of possible
partially-upgraded systems, and realistically we can never test them all;
but I think it's best if we try to avoid knowingly making this worse.
Finally, there is a reason to want this that is circular in nature: if we
want Debian 12 packages to be installable onto non-merged-/usr systems,
and we want to be able to say with reasonable confidence that we think
they work in that situation, then we need to be able to test that they
do, in fact, work. As mentioned under the "Testing and QA" heading,
if we do not keep it possible to force autopkgtest VMs/containers,
piuparts chroots, reproducible-builds chroots and manual-test VMs (or
even scratch installations on real hardware) to be non-merged-/usr,
then we will have trouble testing that. However, I have tried to make it
clear that if a developer sets up such a system, they cannot expect to
be able to upgrade it cleanly to Debian 13, or to a post-Debian-12
state of testing/unstable.
> (2) Some people on -devel would seem to think that we can have smooth
> upgrades from bullseye to bookworm without requiring support for
> non-merged-/usr in every single package in bookworm, i.e., without
> supporting non-merged-/usr for testing/unstable installations during the
> current release cycle.
Some people on -devel have argued that it would be OK to require use of
a special upgrade tool analogous to Ubuntu's do-release-upgrade just this
once, or that it would be OK to require a particular upgrade sequence. For
example, we could ask users to install the usrmerge package first, and
then upgrade; or if dpkg is modified to special-case the merged-/usr
aliasing symlinks, then we might ask users to upgrade dpkg first, and
then upgrade the rest of the system.
I think there is considerable anecdotal evidence that many Debian users
do not read the release notes, and if we ask them to carry out an upgrade
procedure more complicated than
$editor /etc/apt/sources.list && apt update && apt full-upgrade
they will often ignore that request and still expect their upgrade to
work (because in practice it often does, and we've been training users
to expect that for years). That makes me reluctant to require a special
upgrade procedure if it is not strictly necessary.
Another route that I think I have seen suggested is for individual packages
to indicate their lack of support for non-merged-/usr systems by adding a
Depends (or even Pre-Depends?) on usrmerge. If this is part of the
transition route that gets implemented, then it does not contradict what I
said in the draft resolution: I made sure to say that "all packages in
Debian 12 should be installable onto a non-merged-/usr system
*along with their dependencies*". If one of those dependencies is usrmerge,
then the non-merged-/usr system will become a merged-/usr system during
the upgrade, and no constraints have been broken.
I am a little concerned that usrmerge is doing more intrusive surgery on
a running system than even what's normal for an apt/dpkg-based upgrade,
so I would prefer not to rule out designs that defer this action to a
later time. For example, one route that I've seen suggested on -devel
(and even suggested myself sometimes) is that the /usr merge operation
should be done from the initramfs, which effectively means it happens
at the next reboot, when you reboot from the mostly-upgraded state of
"Debian 12 on a Debian 11 kernel" to a fully Debian 12 system. If that
might be the transition plan, then we need to require all packages to
work at least well enough on a non-merged-/usr system to be able to
finish the upgrade and reboot.
Some people do major-release upgrades from inside a full GUI (again,
perhaps they shouldn't, but they do) so the bar for "well enough to
finish the upgrade and reboot" can be quite high: for instance, during
the stretch freeze I had to insert a hack into gdm3 to make sure that
if the user was doing the upgrade in a GNOME desktop, and allowed the
screensaver to lock, then it would remain possible to unlock it again,
despite the private IPC protocol between libgdm and gdm having changed
incompatibly between jessie and stretch.
> What smcv has been arguing for is the most conservative option. But
> which of the following is it the TC wants to say:
>
> - we are sure that this is the only way to ensure smooth upgrades, such
> that it pretty much follows from our previous decision on merged-/usr
>
> - we think there might be viable alternatives to requiring every package
> in bookworm to work on both layouts, but we aren't sure they are safe
> enough, so we're recommending a simple and conservative approach
>
> - we think that ensuring non-merged-/usr testing/unstable installations
> work during this release cycle is reason enough to take this highly
> conservative approach
I'm honestly not sure which of these is "the" reason why I'm recommending
the conservative approach. Some combination of your second and third
points, I suppose?
What do others in the committee think?
smcv
Reply to: