I'm calling for votes on the following resolution as formal advice from
the Technical Committee (Constitution §6.1.5). There are no non-accepted
amendments, so the options to vote on are "yes" or "further discussion".
~~~~ begin text to be voted on ~~~~
Summary
=======
There are currently Debian 11 installations with both the merged-/usr and
non-merged-/usr filesystem layouts. All of these installations should
successfully upgrade via normal upgrade paths to a merged-/usr Debian 12.
Only after the release of Debian 12 can packages assume that all
installations will be merged-/usr.
Main points:
- We have recommended merged-/usr for Debian 12.
- Moving individual files is not merged-/usr.
- "Symlink farms" are not merged-/usr.
- Upgrading a non-merged-/usr system to Debian 12 needs to work.
- Upgrading a merged-/usr system to Debian 12 needs to work.
- Packages cannot assume all systems are merged-/usr until the Debian 13
development cycle begins.
- Upgrading via apt in the usual way should work.
- Testing and QA systems should be able to avoid this transition, but if
they do, they cannot be upgraded beyond Debian 12.
- A package building incorrectly on a non-merged-/usr system is a bug.
- A package building incorrectly on a merged-/usr system is a bug.
- Please stop moving individual packages' files from the root filesystem
into /usr, at least for now.
Definitions and current status
==============================
libQUAL refers to the directories described in FHS 3.0 section 3.10 [1],
such as lib64 on the amd64 architecture.
Merged /usr is the filesystem layout in which /bin, /sbin, /lib and each
/libQUAL that exists are symbolic links to the corresponding directories
below /usr. This results in aliasing between /bin and /usr/bin, and
so on.
In the merged-/usr layout, files whose canonical logical location is
in one of the affected directories on the root filesystem, such as
/bin/bash, /sbin/fsck and /lib/ld-linux.so.2, are physically located at
the corresponding path below /usr, such as /usr/bin/bash. Each file in
one of the affected directories is accessible via two paths: its canonical
logical location (such as /bin/bash or /usr/bin/env), and the other path
implied by the aliasing (such as /usr/bin/bash or /bin/env).
There are two supported categories of Debian 11 installation, which are
currently considered equally-supported:
- Merged-/usr installations. These were installed with debian-installer
from Debian 10 or later, or installed with debootstrap --merged-usr,
or converted from the non-merged-/usr layout by installing the usrmerge
package, or installed or converted by any similar procedure. They have
the merged-/usr layout.
- Non-merged-/usr installations. These were installed with debian-installer
from Debian 9 or earlier and subsequently upgraded without converting
to merged-/usr, or installed with debootstrap --no-merged-usr, or
converted from the merged-/usr layout with dpkg's "dpkg-fsys-usrunmess"
utility or any similar procedure. They have the traditional,
non-merged-/usr layout in which /bin/bash and /usr/bin/env have exactly
those physical paths, and /usr/bin/bash and /bin/env do not exist.
Merged-/usr is not the only filesystem layout that has been proposed for
unifying the root filesystem with /usr. For avoidance of doubt, we do not
consider other filesystem layouts to be implementations of merged-/usr.
In particular, we do not consider these to be implementations of merged-/usr:
- all affected files physically located in /bin, /sbin, /lib and /libQUAL,
with /usr/bin as a symlink to /bin, etc. (this is the reverse of
merged-/usr, and was historically used in the hurd-i386 port)
- a "symlink farm" in each of /bin, /sbin, /lib, /libQUAL with individual
symbolic links such as /bin/bash -> /usr/bin/bash for only those files that
historically had their canonical logical location on the root filesystem
- a "symlink farm" in each of /bin, /sbin, /lib, /libQUAL with individual
symbolic links such as /bin/env -> /usr/bin/env for all files in the
affected directories, regardless of whether they historically had
their canonical logical location on the root filesystem
[1]: https://www.debian.org/doc/packaging-manuals/fhs/fhs-3.0.html#libltqualgtAlternateFormatEssential
Upgrade path from Debian 11 to Debian 12
========================================
The technical committee resolved in #978636 [2] that Debian 12 'bookworm'
should only support the merged-/usr layout. This resolution describes
the implications of that decision in more detail.
Debian installations have traditionally had a straightforward upgrade
path between consecutive releases, and the technical committee expects
maintainers to continue this. In the case of #978636, the upgrades we
are interested in are:
- Upgrading from Debian 11 (stable release) to Debian 12 (stable release)
- Upgrading from Debian 11 (stable release) to testing/unstable during the
development cycle that will lead to Debian 12, and subsequently upgrading
from testing/unstable to Debian 12 (stable release)
What we understand this to imply is as follows:
- Because Debian 11 installations with the merged-/usr layout already
exist, and because Debian 12 should only support the merged-/usr layout,
all packages in Debian 12 should be installable onto a merged-/usr system
along with their dependencies, and work correctly on the resulting system.
- Because Debian 11 installations with the non-merged-/usr layout already
exist, all packages in Debian 12 should be installable onto a non-merged-/usr
system along with their dependencies, and work correctly on the resulting
system.
+ The key reason for this is that apt is not required to perform the
upgrade between stable releases in any particular order, so long
as package dependency relationships are respected; therefore the
upgrade can happen in whatever order apt chooses, which can vary
between machines. Debian has not traditionally required use of a
special upgrade tool similar to Ubuntu's do-release-upgrade(8) and
we believe the upgrade to Debian 12 should be no different (see
below for more details on this topic).
+ Another reason for this is that during the development of Debian 12,
testing/unstable systems undergo a series of partial upgrades, which
similarly will happen in an undefined order.
+ We do not require that the resulting system remains non-merged-/usr:
if the packages involved in this installation transaction are part of
the implementation of a transition to merged-/usr, then installing them
might result in the system becoming merged-/usr.
- The same expectations apply to packages uploaded to testing/unstable
during the development cycle that will lead to Debian 12.
- Debian contributors who are interested in merged-/usr are invited to
implement a straightforward migration path from non-merged-/usr to
merged-/usr. The Technical Committee will not design this migration path
in detail. If disputes arise related to this migration path, or if
advice on this migration path is requested, we will resolve those by
following our usual procedures.
+ One example of a migration path that might be used is for an Essential
package to add a dependency on the usrmerge package, so that it will
be installed automatically during upgrades. We do not require this to
be the migration path that is chosen; it is presented here merely to
demonstrate that such a migration path can exist.
- After Debian 12 is released, the transition to merged-/usr is considered
to be complete. Because we do not support "skipping a release" when
upgrading systems, new packages uploaded to testing/unstable during the
development cycle that will lead to Debian 13 may assume and require
that the system onto which they are installed is merged-/usr.
If a suitable transition mechanism is not available by the time the
Debian 12 freeze is reached, the Technical Committee will rescind our
advice in #978636 and modify our advice in this resolution to reflect
the situation that has been reached. We hope that this will not be necessary.
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636
Use of a special upgrade path
=============================
Some developers have argued that we should deploy merged /usr by means of
a special upgrade path, with all other upgrade paths being declared to be
unsupported. Some examples of special upgrade paths that might be proposed:
- upgrading dpkg before carrying out the rest of the upgrade
- installing the usrmerge package before carrying out the upgrade
- using a special tool similar to Ubuntu's do-release-upgrade(8) which
encapsulates whatever steps are necessary
This would imply that upgrade paths other than the recommended one are not
expected to work.
We are aware that many Debian users do not read the release notes before
upgrading from oldstable to stable, and will expect to be able to upgrade
via whatever apt commands they are used to using, regardless of whether
the release notes recommend something different. Given that there are
alternatives available, we do not think that merged-/usr is sufficiently
good cause to interfere with this.
During the Debian 12 development cycle, users of testing/unstable will also
need an upgrade path from Debian 11 to testing/unstable, or from older to
newer snapshots of testing/unstable. In general this will be done using
apt, and it needs to continue to work if at all possible, even before a
special upgrade tool has been prepared; introducing a "flag day" at which
all testing/unstable users are expected to carry out additional
non-automatic upgrade steps would be disruptive, and in practice many
testing/unstable users are likely to skip those steps.
For these reasons, we make the simple, conservative recommendation that a
special upgrade path should not be required, and upgrading via apt in
approximately the same way as previous Debian releases should continue to work
in general.
Testing and QA
==============
We anticipate that during the development cycle that will lead to Debian 12,
it is likely to be useful for testing and QA infrastructure (such as
autopkgtest, piuparts and/or reproducible-builds) to be able to produce
an installation of Debian testing/unstable that is not merged-/usr,
in order to be able to verify that packages targeted for inclusion in
Debian 12 can still be installed and/or built successfully in a
non-merged-/usr environment during partial upgrades.
As a result, we recommend that if there is an automatic migration from
non-merged-/usr to merged-/usr, it should be possible to prevent that
migration. However, systems where that migration has been prevented are
not required to be fully-supported, and in particular, upgrading them
to Debian 13 or to the state of testing/unstable during development of
Debian 13 should be considered unsupported.
Building packages
=================
In #914897 [3] the Technical Committee resolved that for Debian 11, packages
can validly be built in either a merged-/usr or non-merged-/usr environment.
We understand this to imply that packages built in either a merged-/usr or
non-merged-/usr environment should work as intended in either a merged-/usr
or non-merged-/usr environment.
There is a class of bugs in which a package embeds the absolute path to
some executable or other file, in such a way that if it is built in an
environment with a unified /usr (either via merged-/usr or symlink farms),
this can result in embedding a non-canonical path such as /usr/bin/sh or
/bin/env which will not work correctly in a non-merged-/usr environment.
The Technical Committee reminds maintainers that packages in Debian
12 are expected to work correctly in a non-merged-/usr environment,
and therefore this class of bugs needs to be resolved. As a result, we
recommend that these bugs should generally be treated as release-critical
for Debian 12 by maintainers and the release team.
The Reproducible Builds effort tracks this class of bugs as
"paths_vary_due_to_usrmerge" [4] and many bugs in this class are
tracked with the usertag "usrmerge" [5]. In Autotools-based
build systems, they can often be avoided by passing arguments such as
`SED=/bin/sed` to the configure script.
Note that although the name of the usertag mentions usrmerge, this class
of bugs is not specific to the usrmerge package, and not completely specific
to merged-/usr; many of these bugs would also manifest if unified /usr was
achieved via symlink farms, or if a local executable such as
/usr/local/bin/sed existed on the build system.
#993675 [6] is a typical example of this class of bugs.
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914897
[4]: https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html
[5]: https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=reproducible-builds%40lists.alioth.debian.org&tag=usrmerge
[6]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993675
Moratorium on moving files' logical locations into /usr
=======================================================
In the past, some package maintainers have taken advantage of the
fact that /usr is now required to be mounted during early boot
(since Debian 9) to move individual files from the root filesystem
to /usr. Examples of this being done during previous development cycles
include the libglib-2.0.so.0 and libgcrypt.so.20 shared libraries,
and the command-line utilities in the iptables package.
The Technical Committee recommends that during the Debian 12 development
cycle, the maintainers of individual packages should not proactively move
files from the root filesystem to the corresponding locations in /usr in
the data.tar.* of packages. Files that were in /usr in the Debian 11
release should remain in /usr, while files that were in /bin, /lib* or
/sbin in the Debian 11 release should remain in those directories.
If files were moved from /bin, /lib* or /sbin into /usr since the
Debian 11 release, they should be moved back to their Debian 11
locations.
Similarly, during the Debian 12 development cycle, we recommend that
maintainers of tools such as debhelper should not move files from the
root filesystem into /usr.
For example, debhelper 13.4 started moving systemd units from /lib/systemd
into /usr/lib/systemd, but this was subsequently reverted in 13.5.2. We
consider the revert that was done in 13.5.2 to have been an appropriate
course of action.
We issue this recommendation for several reasons:
- The transition to merged-/usr will make the logical locations of these
files irrelevant to their physical locations.
- The transitional mechanisms necessary to prevent such moves from breaking
other packages that hard-code specific paths are error-prone, and can
themselves interfere with the transition to merged-/usr.
- After the transition to merged-/usr has completed, during the Debian 13
development cycle, we expect that maintainers will be able to move these
files without needing transitional mechanisms.
- On merged-/usr systems, there is a possible failure mode involving files
being moved between packages (with Replaces) during the same release
cycle that their logical location is changed from the root filesystem
to the corresponding aliased directory in /usr, which can result in
the affected file disappearing. This can be avoided by not changing
the file's logical location until the beginning of the Debian 13
development cycle, after the transition to merged-/usr is complete.
- On non-merged-/usr systems, a failure mode has been observed in which
older shared libraries in /lib/MULTIARCH are not always deleted as
intended, and interfere with correct loading of the newer shared
library in /usr/lib/MULTIARCH. This can be avoided by not changing
the file's logical location until the beginning of the Debian 13
development cycle, after the transition to merged-/usr is complete,
at which point ldconfig(8) will choose the newer library even if
this occurs.
This recommendation applies until Debian 12 is released, or until
a subsequent Technical Committee resolution rescinds it, whichever
is sooner. We intend to rescind this recommendation if mechanisms are
developed to avoid the undesired side-effects of moving files from the
root filesystem into /usr.
~~~~ end text to be voted on ~~~~
Attachment:
signature.asc
Description: PGP signature