Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib
On Sun, 2020-02-16 at 11:59:56 +0000, Simon McVittie wrote:
> I would be grateful if people who advocate transitioning individual
> packages, and people who consider the approach taken by usrmerge and
> debootstrap to be sufficient, could refer to their preferred route in a
> way that makes it clear which one they are advocating. Saying we should
> do a transition "properly" is tautologous - of course we should! - but
> when people disagree about what the proper way to do it is, it becomes an
> ambiguous recommendation that doesn't guide anyone to do the right thing.
I've been consistently calling the concept of merging /* into /usr/* as
merged-/usr and the specific approach of using directory symlinks as
merged-/usr-via-symlinks (although I think that's confusing as the
other approach does use symlink farms), so I think using either
merged-/usr-via-aliased-dirs or merged-/usr-via-symlink-dirs is more
clear (will be renaming the buildinfo tainted tag). The approach I've
been proposing I'd call merged-/usr-via-moves-and-symlink-farms or
something along those lines.
I've dumped this all into <https://wiki.debian.org/Teams/Dpkg/MergedUsr>
and updated the Dpkg FAQ:
<https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F>
> The approach that transitions individual packages solves some bugs
> (notably, if you ask dpkg which package owns /usr/lib/MA/libfoo.so.0,
> you get the right answer, which has a lot of desirable consequences). It
> also appears to *cause* some bugs. The ones I know about are:
>
> * packages that hard-code paths into /lib stop working on systems that
> have *not* undergone the usrmerge-style /usr merge, because
> /lib/MA/libfoo.so.0 no longer exists (for example see #950715
> involving libgcc.so.1 and the gold linker);
For any pathname that has been hardcoded a symlink can be used for
backwards compat, nothing unlike /bin or /sbin here. This looks just
like a normal bug from a botched transition, nothing special.
> * there is a class of bugs on systems that have *not* undergone the
> usrmerge-style /usr merge, involving old libraries lingering in /lib/MA
> (see #949395 for a summary of the instances that I know about), which
> are very hard to debug because they are unreproducible, to do with the
> state of an individual system, and are related to upgrades that happened
> years in the past and whose logs expired long ago;
merged-/usr (in general) seems irrelevant here, if this is really a bug
in dpkg, then it needs fixing regardless of any approach taken, as there
is nothing in dpkg really distinguishing between / and /usr as it is
pathname neutral.
I've pending to comment on that bug, but I was skimming over the involved
code and didn't see anything obviously wrong that would forget pathnames.
But my first suspect would be that we are not doing fsync() on the parent
directories, the second would be perhaps related to multiarch refcounting
logic, but ISTR this having happened in old glibc perhaps before multiarch
was deployed, but would need to check.
> * when paths migrate between package names and between paths at the same
> time, there can be undetected file conflicts on systems that *have*
> undergone the usrmerge-style /usr merge (for example see #950624,
> again in libgcc.so.1)
This is one of the issues inherent in merged-/usr-via-aliased-dirs.
> In Debian we have (as usual) made life more difficult for ourselves by the
> /usr merge being optional,
I fully agree, but not on the reason for the root cause. Allowing the
merged-/usr-via-aliased-dirs hack at all, has meant that we cannot do
fully automatic migrations via debhelper with no maintainer scripts
involved whatsoever. :(
> which means that in any transition or upgrade
> scenario, maintainers have to consider two cases: one where the system
> has undergone the /usr merge, and one where it has not. For example,
> #950624 only happens on systems that have undergone the /usr merge,
> while #950715 and #949395 only happen on systems that have not. Testing
> on any single system cannot detect all of these: to detect all the bugs,
> we have to try both configurations.
The difference between these issues IMO is one between bugs (a botched
migration with no compat symlinks, and a potential dpkg bug), and design
flaws inherent in the merged-/usr-via-aliased-dirs approach.
I'm not sure how anyone could claim that the merged-/usr-via-aliased-dirs
approach is not a hack, even if it might have some possible benefits,
when it circumvents the package manager completely (we might as well
ditch it and go use bare tarballs or something :/…), and implies action
at a distance where the filesystem layout is injected from something
that is not packaged at all and should not be packaged (or we'd be forced
to hardcode bootstrapping ordering into each bootstrapper forever) so we
also lose locality from the .deb.
That approach would be acceptable in a distant future once and iff we ship
no objects under top root directories, which for now would imply breaking
ABI/APIs. Doing the proposed mandatory merged-/usr-via-aliased-dirs for a
release cycle, and moving everything en mass, still does not fix the
aliasing problems for all the paths that are hardcoded on non-/usr
locations. This would mean painting ourselves into an even tinier and
messier corner than where we are now…
IMO the technically better and more sound solution would be to devise
a way to revert the mess caused by the merged-/usr-via-aliased-dirs
approach, and then convert everything automatically via debhelper.
Thanks,
Guillem
Reply to: