Dear Technical Committee members,
(CC'ed to submitter, and debootstrap maintainers for information and feedack)
Here's the current state of the draft resolution; which `master` is at
https://salsa.debian.org/debian/tech-ctte/blob/master/914897_merged_usr/ballot.md
I will submit it to vote on Friday 22.; the discussion period is open!
# #914897: tech-ctte: Should debootstrap disable merged `/usr` by default?
## What is "merged `/usr`"
"Merged `/usr`" describes a possible future standard directories scheme in
which the `/{bin,sbin,lib*}/` directories have been made superfluous through
replacing them by symlinks to their `/usr` equivalents (/usr/{bin,sbin,lib*}).
The motivation to get Debian systems to converge towards such a scheme is
vastly documented elsewhere ([FDO's TheCaseForTheUsrMerge][0],
[wiki.d.o UsrMerge][1]) but can be summarized as the following points:
* having separate `/` and `/usr` filesystems has been useful in the past for
booting without initramfs onto a minimal root filesystem that carried just
enough to mount the `/usr` filesystem later in the boot process. Given the
evolution of physical hosts' capabilities, initramfs'es have been default in
Debian (and elsewhere) for a long time, and most systems no longer have an
intermediate state during boot in which they have only `/`, but not `/usr`,
mounted.
* another use-case is to be able to share an identical `/usr` over a network
link; hence booting an initramfs, mounting a local `/`, then mounting `/usr`
over the network. It seems that an initramfs with everything needed to mount
a filesystem over a network link directly actually has a smaller footprint.
* booting with `/` only is not systematically tested in Debian anymore;
* the packaging infrastructure to install files outside of `/usr` is not
standard and represents technical debt:
* given its status as remnant "folklore", the distinction between what _needs_
to be shipped in `/` and what can stay in `/usr` is often interpreted
arbitrarily;
* allowing shipment of identically-named libraries or binaries in different
paths can confuse common understanding of paths precedence.
The arguments against moving the base directories' scheme towards
"merged `/usr`" are as follows:
* there's no gain in disrupting something that is not inherently broken;
* `/{bin,sbin,lib*}/` → `/usr/{bin,sbin,lib*}/` symlinks create confusing views
of the system (`/bin/cat` and `/usr/bin/cat` are the same file), and dpkg
doesn't support this situation cleanly
[#134758](https://bugs.debian.org/134758).
[0]: https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
[1]: https://wiki.debian.org/UsrMerge
The compatibility symbolic links `/lib` → `/usr/lib` and
`/lib64` → `/usr/lib64` are required by the various CPUs' platform ABIs (for
example i386 requires `/lib/ld-linux.so.2` to resolve to glibc's `ld.so`, and
amd64 requires `/lib64/ld-linux-x86-64.so.2`) so there are no plans to remove
them altogether. Similarly, removing `/bin` is not under consideration because
it would break the assumption that `/bin/sh` exists, and removing `/sbin` would
break the assumption that `/sbin/fsck.*` and `/sbin/mount.*` exist.
## "merged `/usr`" in Debian
In Debian buster, the current testing suite, "merged `/usr`" is only considered
for implementation with symlinks (there are no proposals for simply dropping
`/{bin,sbin,lib*}`) and is implemented in two main ways:
* existing hosts can be made to have a "merged `/usr`" by installing the
[usrmerge][2] package;
* new hosts get the `/{bin,sbin,lib*}/`→ `/usr/{bin,sbin,lib*}/` symlinks by
default when using debootstrap >= 1.0.102.
The usrmerge package contains a `/usr/lib/convert-usrmerge` perl executable
that runs in `postinst`, that will move the contents of `/{bin,sbin,lib*}/` and
replace these directories with symlinks when empty.
It is also possible to merge `/usr` in other ways, for example with
`debootstrap --merged-usr` or by bootstrapping into a chroot that already
contains the necessary symlinks.
[2]: https://tracker.debian.org/pkg/usrmerge
## Issues from "merged `/usr`"
Since the default change in debootstrap 1.0.102, some issues have arisen.
Due to the fact that some buster/sid hosts have the "merged `/usr`" symlinks in
place, it has been observed that some binary packages carried some traces of
these differences (notably official packages built on Debian buildd hosts which
had been resetup).
Some such differences can actually render the built packages unuseable on
non-"merged `/usr`" systems.
For example, if `cat` is detected at build-time in `/usr/bin/cat` (where
coreutils ships `/bin/cat`), a binary hardcoding that path will try to use
`/usr/bin/cat` after installation, but that path doesn't exist in
non-"merged `/usr`" systems.
In order to mitigate this, debootstrap has been modified to let its "buildd"
variant be non-"merged `/usr`", the Debian buildds have been resetup and the
affected packages rebuilt.
The lesson here is that with the existance of (any of) the usrmerge and the
debootstrap default change, "merged `/usr`" Debian systems exist already, and
that packages built on hosts with such directory schemes can _potentially_ be
broken on non-"merged `/usr`" systems.
At this point, the two variants have to be supported, at least as installation
targets of Debian packages.
Two initiatives are worth mentioning at this point:
* [a patch](https://lists.debian.org/20181202212535.GC11687@gaara.hadrons.org)
has been proposed for dpkg-buildpackage to mark packages built on
"merged `/usr`" hosts with a `Build-Tainted-By: merged-usr-via-symlinks`;
* the reproducible builds team has added a "merged `/usr`" variation to their
setup, and have then
[tagged](https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html)
the Debian packages from unstable which had differences due to
"merged `/usr`". It seems that ~61 packages were affected by differing
builds; 32 from these have been fixed in unstable already.
## The long-term desireable situation
Various valid long-term desireable situations coexist, and while discussing
immediate countermeasures, it is useful to keep the long-term outcome that
those are most likely to produce.
These are the five possible situations at the time of bullseye (buster + 1):
* `none`: "merged `/usr`" has been reverted
* `weak`: both directory schemes are allowed, packages only built on classical
hosts
* `middle`: both directory schemes are allowed, packages can be built anywhere
* `hard`: both directory schemes are allowed, packages only built on
"merged `/usr`" hosts
* `all`: only "merged `/usr`" directory schemes are allowed, packages only
built on "merged `/usr`" hosts
It can be summarized by the following table:
```
| | Host types that are allowed | Are merged `/usr` | Official packages are built on | Packages built on … can break on the other |
| Codename | classical hosts | merged `/usr` hosts | symlinks allowed | classical hosts | merged `/usr` hosts | classical hosts | merged `/usr` hosts |
|----------|-----------------|---------------------|-------------------|—----------------|---------------------|---------------------|----------------------|
| none | yes | no | no | yes | no | yes | yes |
| weak | yes | yes | yes | yes | no | no | yes |
| middle | yes | yes | yes | yes | yes | no | no |
| hard | yes | yes | yes | no | yes | no | no |
| all | no | yes | yes | no | yes | yes | no |
```
The current state of buster is `weak`.
=== DRAFT Resolution ===
The Technical Committee resolves to:
* Option A: Ask the debootstrap maintainers to disable "merged `/usr`" by
default
(Using its §6.1.4 "Overrule a Developer" power; requires a 3:1 majority)
Given that:
* hosts with both directory schemes already exist,
* the "merged `/usr`" directory scheme ought to be reserved for special
use-cases,
* official packages ought to only be built on classical directory schemes,
… the Technical Committee considers that the desireable solution at the time
of bullseye is `weak`; and asks the debootstrap maintainers to disable
"merged `/usr`" by default.
* Option B: Decline to override the debootstrap maintainers; offer advice
(Using its §6.1.5 "Offering advice" power)
Given that:
* hosts with both directory schemes already exist,
* it seems unpractical to allow official packages to be built on either
directory schemes,
* there's inherent value in the simplicity of "merged `/usr`" directory
schemes,
… the Technical Committee considers that the desireable solution at the time
of bullseye is `hard`; and declines to override the debootstrap maintainers.
=== End DRAFT Resolution ===
Cheers,
OdyXAttachment:
signature.asc
Description: This is a digitally signed message part.