Re: A summary of where I think we are on the technical side of the merged /usr discussion
On Tue, 17 Aug 2021 at 12:59:21 -0400, Theodore Ts'o wrote:
> > Such packages are already violating a Policy "should", because they're
> > not building reproducibly (and the reproducible-builds infra tests this
> > for testing and unstable).
>
> Do we have a dashboard for this so the list of which source packages
> result in different binary packages depending on whether they are
> built with a usrmerge vs !usrmerge system? We could look at the
> reproducible-builds reports, but not all reproducible build failures
> are caused by the usrmerge/!usrmerge dependency, right?
Sort of. For packages where someone from the reproducible-builds team
has done the analysis,
https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html
and similar pages for other suites.
Additionally,
https://tests.reproducible-builds.org/debian/unstable/amd64/index_no_notes.html
and similar pages list packages that have not been analyzed or categorized,
and *might* have this issue.
> On a separate question, at the moment e2fsprogs is installing some
> files in /sbin and /lib, and other in /usr/sbin and /usr/lib, etc.,
> since historically the goal was to allow systems to boot, and bring up
> networking, etc., without /usr being mounted.
Right. Booting without /usr is unsupported since buster, but if your
package was traditionally split between the rootfs and /usr, the
conservative thing to do is to leave it as-is.
> As a result there are some breakout lintain warnings:
>
> W: libext2fs-dev: breakout-link usr/lib/x86_64-linux-gnu/libe2p.so -> lib/x86_64-linux-gnu/libe2p.so.2
> W: libext2fs-dev: breakout-link usr/lib/x86_64-linux-gnu/libext2fs.so -> lib/x86_64-linux-gnu/libext2fs.so.2
> W: ss-dev: breakout-link usr/lib/x86_64-linux-gnu/libss.so -> lib/x86_64-linux-gnu/libss.so.2
I don't know what the purpose of the breakout-link tag is, to be honest.
It seems like it's often a false-positive, and I don't think these links
are bugs.
> Suppose I released a new version of e2fsprogs targetting sid and
> bookworm which installs everything in /usr/bin, /usr/sbin, /usr/lib,
> etc, instead of splitting up files in /... and /usr/...
>
> * Is this a desirable thing to do now? (Post-bullseye release)
Maybe.
If a distro-wide migration to merged-/usr is genuinely happening (and the
project isn't going to hold a GR to overrule the Technical Committee or
anything like that), then one point of view says the pragmatic thing to
do might be to leave this package as-is until the bookworm+1 cycle opens,
and then simplify it by taking advantage of merged-/usr being the only
supported filesystem layout; moving the files around is a potential
source of bugs, and the transition to merged-/usr will resolve it for
you with less work.
However, some people (most notably the dpkg maintainer, who has thought
about this more than most) argue that merged-/usr's "aliasing" symlinks
/bin -> usr/bin, etc. are unsupportable, and the only correct way to
consolidate static files to be physically located under /usr is to
gradually build up symlink farms below /bin and so on. If people with this
point of view turn out to represent project consensus, or if they do not
represent consensus but are sufficiently loud that they delay merged-/usr
indefinitely, then waiting for merged-/usr to solve the problem for you
might not be viable.
If your opinion is somewhere in between those extremes, then you might be
willing to do the work of migrating files from the root filesystem into
/usr in order to hedge your bets, even though you expect a more general
distro-wide move to merged-/usr to make the distinction irrelevant in
practice. This is what has happened in dbus and glib2.0.
> * What are the potential risks of doing this now?
For users of non-merged-/usr
----------------------------
If components of your package implement a third-party filesystem "API",
then you need to check that the consumer is going to look in both the
rootfs and /usr. For e2fsprogs, I would expect the problem areas to be
the /sbin/fsck.TYPE and /sbin/mkfs.TYPE interfaces: if you install
to /usr/sbin/fsck.TYPE and /usr/sbin/mkfs.TYPE, will the fsck and mkfs
wrappers in util-linux still find them?
If paths from your package are hard-coded somewhere, and are no longer
the physical location of the file, then you will need a compat symlink.
Because merged-/usr exists, you cannot ship both /foo and /usr/foo in
the data.tar.*; you will have to create the compat symlink from the
maintainer scripts. The handling of /{usr/,}bin/touch in coreutils is a
nice simple example. This is most likely to affect executables, because
the paths to shared libraries are not usually hard-coded.
If your shared libraries somehow hit the same mysterious bug discussed
in #911225 and #949395, with the old libraries not getting removed
from their path on the root filesystem, then you might have to apply
the same workaround we did in GLib (GLib has a generic script you can
copy for this, at your own risk). This failure mode has been seen in GLib
(multiple times) and libcrypt (once). I still have no idea how it happens,
or why it affected GLib more than other packages...
For users of merged-/usr
------------------------
If you get the maintainer-script glue for compat symlinks wrong, then
the package might become uninstallable or otherwise broken on merged-/usr
systems. autopkgtest and piuparts can help to detect this.
If merged-/usr does indeed become mandatory for bookworm as the Technical
Committe have recommended, then you might feel that the work you've done
on disentangling this was a waste of effort, and you might wish you had
left the package as-is until the beginning of the bookworm+1 cycle and
then drastically simplified it by taking advantage of merged-/usr being
the only supported layout.
Generally
---------
For completeness:
If the project as a whole reaches consensus that the Technical Committee
have been consistently wrong about merged-/usr since late 2018, and
overrules us via a GR, then you might have to change how this works.
Or, if someone finds a showstopper issue with merged-/usr that has not
been a serious practical problem in buster, bullseye, Ubuntu, Fedora or
Arch but turns out to be sufficiently bad to make the TC change our minds,
then similarly you might have to change how this works.
> * Bullseye users might still be assuming that they can boot w/o
> /usr mounted, is that correct?
No, this has been officially unsupported since buster. (Perhaps users
still have this assumption, but if they do, they're wrong.)
smcv
Reply to: