Hello,
On Wed 13 Oct 2021 at 08:13PM +01, Simon McVittie wrote:
> I'm calling for votes on the following resolution as formal advice from
> the Technical Committee (Constitution §6.1.5). There are no non-accepted
> amendments, so the options to vote on are "yes" or "further discussion".
I vote
yes > further discussion
> ~~~~ begin text to be voted on ~~~~
>
> Summary
> =======
>
> There are currently Debian 11 installations with both the merged-/usr and
> non-merged-/usr filesystem layouts. All of these installations should
> successfully upgrade via normal upgrade paths to a merged-/usr Debian 12.
> Only after the release of Debian 12 can packages assume that all
> installations will be merged-/usr.
>
> Main points:
>
> - We have recommended merged-/usr for Debian 12.
> - Moving individual files is not merged-/usr.
> - "Symlink farms" are not merged-/usr.
> - Upgrading a non-merged-/usr system to Debian 12 needs to work.
> - Upgrading a merged-/usr system to Debian 12 needs to work.
> - Packages cannot assume all systems are merged-/usr until the Debian 13
> development cycle begins.
> - Upgrading via apt in the usual way should work.
> - Testing and QA systems should be able to avoid this transition, but if
> they do, they cannot be upgraded beyond Debian 12.
> - A package building incorrectly on a non-merged-/usr system is a bug.
> - A package building incorrectly on a merged-/usr system is a bug.
> - Please stop moving individual packages' files from the root filesystem
> into /usr, at least for now.
>
> Definitions and current status
> ==============================
>
> libQUAL refers to the directories described in FHS 3.0 section 3.10 [1],
> such as lib64 on the amd64 architecture.
>
> Merged /usr is the filesystem layout in which /bin, /sbin, /lib and each
> /libQUAL that exists are symbolic links to the corresponding directories
> below /usr. This results in aliasing between /bin and /usr/bin, and
> so on.
>
> In the merged-/usr layout, files whose canonical logical location is
> in one of the affected directories on the root filesystem, such as
> /bin/bash, /sbin/fsck and /lib/ld-linux.so.2, are physically located at
> the corresponding path below /usr, such as /usr/bin/bash. Each file in
> one of the affected directories is accessible via two paths: its canonical
> logical location (such as /bin/bash or /usr/bin/env), and the other path
> implied by the aliasing (such as /usr/bin/bash or /bin/env).
>
> There are two supported categories of Debian 11 installation, which are
> currently considered equally-supported:
>
> - Merged-/usr installations. These were installed with debian-installer
> from Debian 10 or later, or installed with debootstrap --merged-usr,
> or converted from the non-merged-/usr layout by installing the usrmerge
> package, or installed or converted by any similar procedure. They have
> the merged-/usr layout.
>
> - Non-merged-/usr installations. These were installed with debian-installer
> from Debian 9 or earlier and subsequently upgraded without converting
> to merged-/usr, or installed with debootstrap --no-merged-usr, or
> converted from the merged-/usr layout with dpkg's "dpkg-fsys-usrunmess"
> utility or any similar procedure. They have the traditional,
> non-merged-/usr layout in which /bin/bash and /usr/bin/env have exactly
> those physical paths, and /usr/bin/bash and /bin/env do not exist.
>
> Merged-/usr is not the only filesystem layout that has been proposed for
> unifying the root filesystem with /usr. For avoidance of doubt, we do not
> consider other filesystem layouts to be implementations of merged-/usr.
> In particular, we do not consider these to be implementations of merged-/usr:
>
> - all affected files physically located in /bin, /sbin, /lib and /libQUAL,
> with /usr/bin as a symlink to /bin, etc. (this is the reverse of
> merged-/usr, and was historically used in the hurd-i386 port)
>
> - a "symlink farm" in each of /bin, /sbin, /lib, /libQUAL with individual
> symbolic links such as /bin/bash -> /usr/bin/bash for only those files that
> historically had their canonical logical location on the root filesystem
>
> - a "symlink farm" in each of /bin, /sbin, /lib, /libQUAL with individual
> symbolic links such as /bin/env -> /usr/bin/env for all files in the
> affected directories, regardless of whether they historically had
> their canonical logical location on the root filesystem
>
> [1]: https://www.debian.org/doc/packaging-manuals/fhs/fhs-3.0.html#libltqualgtAlternateFormatEssential
>
> Upgrade path from Debian 11 to Debian 12
> ========================================
>
> The technical committee resolved in #978636 [2] that Debian 12 'bookworm'
> should only support the merged-/usr layout. This resolution describes
> the implications of that decision in more detail.
>
> Debian installations have traditionally had a straightforward upgrade
> path between consecutive releases, and the technical committee expects
> maintainers to continue this. In the case of #978636, the upgrades we
> are interested in are:
>
> - Upgrading from Debian 11 (stable release) to Debian 12 (stable release)
>
> - Upgrading from Debian 11 (stable release) to testing/unstable during the
> development cycle that will lead to Debian 12, and subsequently upgrading
> from testing/unstable to Debian 12 (stable release)
>
> What we understand this to imply is as follows:
>
> - Because Debian 11 installations with the merged-/usr layout already
> exist, and because Debian 12 should only support the merged-/usr layout,
> all packages in Debian 12 should be installable onto a merged-/usr system
> along with their dependencies, and work correctly on the resulting system.
>
> - Because Debian 11 installations with the non-merged-/usr layout already
> exist, all packages in Debian 12 should be installable onto a non-merged-/usr
> system along with their dependencies, and work correctly on the resulting
> system.
>
> + The key reason for this is that apt is not required to perform the
> upgrade between stable releases in any particular order, so long
> as package dependency relationships are respected; therefore the
> upgrade can happen in whatever order apt chooses, which can vary
> between machines. Debian has not traditionally required use of a
> special upgrade tool similar to Ubuntu's do-release-upgrade(8) and
> we believe the upgrade to Debian 12 should be no different (see
> below for more details on this topic).
>
> + Another reason for this is that during the development of Debian 12,
> testing/unstable systems undergo a series of partial upgrades, which
> similarly will happen in an undefined order.
>
> + We do not require that the resulting system remains non-merged-/usr:
> if the packages involved in this installation transaction are part of
> the implementation of a transition to merged-/usr, then installing them
> might result in the system becoming merged-/usr.
>
> - The same expectations apply to packages uploaded to testing/unstable
> during the development cycle that will lead to Debian 12.
>
> - Debian contributors who are interested in merged-/usr are invited to
> implement a straightforward migration path from non-merged-/usr to
> merged-/usr. The Technical Committee will not design this migration path
> in detail. If disputes arise related to this migration path, or if
> advice on this migration path is requested, we will resolve those by
> following our usual procedures.
>
> + One example of a migration path that might be used is for an Essential
> package to add a dependency on the usrmerge package, so that it will
> be installed automatically during upgrades. We do not require this to
> be the migration path that is chosen; it is presented here merely to
> demonstrate that such a migration path can exist.
>
> - After Debian 12 is released, the transition to merged-/usr is considered
> to be complete. Because we do not support "skipping a release" when
> upgrading systems, new packages uploaded to testing/unstable during the
> development cycle that will lead to Debian 13 may assume and require
> that the system onto which they are installed is merged-/usr.
>
> If a suitable transition mechanism is not available by the time the
> Debian 12 freeze is reached, the Technical Committee will rescind our
> advice in #978636 and modify our advice in this resolution to reflect
> the situation that has been reached. We hope that this will not be necessary.
>
> [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636
>
> Use of a special upgrade path
> =============================
>
> Some developers have argued that we should deploy merged /usr by means of
> a special upgrade path, with all other upgrade paths being declared to be
> unsupported. Some examples of special upgrade paths that might be proposed:
>
> - upgrading dpkg before carrying out the rest of the upgrade
>
> - installing the usrmerge package before carrying out the upgrade
>
> - using a special tool similar to Ubuntu's do-release-upgrade(8) which
> encapsulates whatever steps are necessary
>
> This would imply that upgrade paths other than the recommended one are not
> expected to work.
>
> We are aware that many Debian users do not read the release notes before
> upgrading from oldstable to stable, and will expect to be able to upgrade
> via whatever apt commands they are used to using, regardless of whether
> the release notes recommend something different. Given that there are
> alternatives available, we do not think that merged-/usr is sufficiently
> good cause to interfere with this.
>
> During the Debian 12 development cycle, users of testing/unstable will also
> need an upgrade path from Debian 11 to testing/unstable, or from older to
> newer snapshots of testing/unstable. In general this will be done using
> apt, and it needs to continue to work if at all possible, even before a
> special upgrade tool has been prepared; introducing a "flag day" at which
> all testing/unstable users are expected to carry out additional
> non-automatic upgrade steps would be disruptive, and in practice many
> testing/unstable users are likely to skip those steps.
>
> For these reasons, we make the simple, conservative recommendation that a
> special upgrade path should not be required, and upgrading via apt in
> approximately the same way as previous Debian releases should continue to work
> in general.
>
> Testing and QA
> ==============
>
> We anticipate that during the development cycle that will lead to Debian 12,
> it is likely to be useful for testing and QA infrastructure (such as
> autopkgtest, piuparts and/or reproducible-builds) to be able to produce
> an installation of Debian testing/unstable that is not merged-/usr,
> in order to be able to verify that packages targeted for inclusion in
> Debian 12 can still be installed and/or built successfully in a
> non-merged-/usr environment during partial upgrades.
>
> As a result, we recommend that if there is an automatic migration from
> non-merged-/usr to merged-/usr, it should be possible to prevent that
> migration. However, systems where that migration has been prevented are
> not required to be fully-supported, and in particular, upgrading them
> to Debian 13 or to the state of testing/unstable during development of
> Debian 13 should be considered unsupported.
>
> Building packages
> =================
>
> In #914897 [3] the Technical Committee resolved that for Debian 11, packages
> can validly be built in either a merged-/usr or non-merged-/usr environment.
> We understand this to imply that packages built in either a merged-/usr or
> non-merged-/usr environment should work as intended in either a merged-/usr
> or non-merged-/usr environment.
>
> There is a class of bugs in which a package embeds the absolute path to
> some executable or other file, in such a way that if it is built in an
> environment with a unified /usr (either via merged-/usr or symlink farms),
> this can result in embedding a non-canonical path such as /usr/bin/sh or
> /bin/env which will not work correctly in a non-merged-/usr environment.
> The Technical Committee reminds maintainers that packages in Debian
> 12 are expected to work correctly in a non-merged-/usr environment,
> and therefore this class of bugs needs to be resolved. As a result, we
> recommend that these bugs should generally be treated as release-critical
> for Debian 12 by maintainers and the release team.
>
> The Reproducible Builds effort tracks this class of bugs as
> "paths_vary_due_to_usrmerge" [4] and many bugs in this class are
> tracked with the usertag "usrmerge" [5]. In Autotools-based
> build systems, they can often be avoided by passing arguments such as
> `SED=/bin/sed` to the configure script.
>
> Note that although the name of the usertag mentions usrmerge, this class
> of bugs is not specific to the usrmerge package, and not completely specific
> to merged-/usr; many of these bugs would also manifest if unified /usr was
> achieved via symlink farms, or if a local executable such as
> /usr/local/bin/sed existed on the build system.
>
> #993675 [6] is a typical example of this class of bugs.
>
> [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914897
> [4]: https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html
> [5]: https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=reproducible-builds%40lists.alioth.debian.org&tag=usrmerge
> [6]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993675
>
> Moratorium on moving files' logical locations into /usr
> =======================================================
>
> In the past, some package maintainers have taken advantage of the
> fact that /usr is now required to be mounted during early boot
> (since Debian 9) to move individual files from the root filesystem
> to /usr. Examples of this being done during previous development cycles
> include the libglib-2.0.so.0 and libgcrypt.so.20 shared libraries,
> and the command-line utilities in the iptables package.
>
> The Technical Committee recommends that during the Debian 12 development
> cycle, the maintainers of individual packages should not proactively move
> files from the root filesystem to the corresponding locations in /usr in
> the data.tar.* of packages. Files that were in /usr in the Debian 11
> release should remain in /usr, while files that were in /bin, /lib* or
> /sbin in the Debian 11 release should remain in those directories.
> If files were moved from /bin, /lib* or /sbin into /usr since the
> Debian 11 release, they should be moved back to their Debian 11
> locations.
>
> Similarly, during the Debian 12 development cycle, we recommend that
> maintainers of tools such as debhelper should not move files from the
> root filesystem into /usr.
>
> For example, debhelper 13.4 started moving systemd units from /lib/systemd
> into /usr/lib/systemd, but this was subsequently reverted in 13.5.2. We
> consider the revert that was done in 13.5.2 to have been an appropriate
> course of action.
>
> We issue this recommendation for several reasons:
>
> - The transition to merged-/usr will make the logical locations of these
> files irrelevant to their physical locations.
>
> - The transitional mechanisms necessary to prevent such moves from breaking
> other packages that hard-code specific paths are error-prone, and can
> themselves interfere with the transition to merged-/usr.
>
> - After the transition to merged-/usr has completed, during the Debian 13
> development cycle, we expect that maintainers will be able to move these
> files without needing transitional mechanisms.
>
> - On merged-/usr systems, there is a possible failure mode involving files
> being moved between packages (with Replaces) during the same release
> cycle that their logical location is changed from the root filesystem
> to the corresponding aliased directory in /usr, which can result in
> the affected file disappearing. This can be avoided by not changing
> the file's logical location until the beginning of the Debian 13
> development cycle, after the transition to merged-/usr is complete.
>
> - On non-merged-/usr systems, a failure mode has been observed in which
> older shared libraries in /lib/MULTIARCH are not always deleted as
> intended, and interfere with correct loading of the newer shared
> library in /usr/lib/MULTIARCH. This can be avoided by not changing
> the file's logical location until the beginning of the Debian 13
> development cycle, after the transition to merged-/usr is complete,
> at which point ldconfig(8) will choose the newer library even if
> this occurs.
>
> This recommendation applies until Debian 12 is released, or until
> a subsequent Technical Committee resolution rescinds it, whichever
> is sooner. We intend to rescind this recommendation if mechanisms are
> developed to avoid the undesired side-effects of moving files from the
> root filesystem into /usr.
>
> ~~~~ end text to be voted on ~~~~
--
Sean Whitton
Attachment:
signature.asc
Description: PGP signature