Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg)
Hi Richard,
On Fri, Jun 09, 2023 at 02:42:25PM -0500, Richard Laager wrote:
> Is the broader context here that this is an alternative to teaching dpkg
> about aliasing? That is, we just arrange the transition correctly such that
> we get out of the aliased situation as part of upgrading to trixie?
Yes, I think the idea that we are mostly exploring now is not teaching
dpkg about aliasing and rather move all the files to their canonical
location such that there no longer is any aliasing that dpkg would have
to deal with.
I don't think this is complete consensus at this time, but the majority
of discussion participants appear to favour this approach.
> Because you want to support non-usr-merged systems, e.g. for derivatives?
dpkg is used in any different contexts. A very simple example of a
non-merged system would be Debian stretch. For another dpkg really is
being used for things that are not based on Debian. While it is the
Debian package manager, it has uses beyond dpkg has (thus far) stayed
away from imposing policy on the filesystem layout.
> They aren't going to want to delete /bin either, so I don't see how a
> special-case preventing deletion of /bin would be problematic.
Indeed. However, if you actually manage to trigger this, it can be very
surprising. Your hard coded list would also contain /lib32, /libx32 and
/libo32. Then you install some mipsen packages, remove them and wonder
why /libo32 does not go away. piuparts is definitely unhappy at this
point. I am quite sure that this behaviour would break something. What I
am not sure about is whether accepting such breakage is a reasonable
trade-off.
> Am I understanding the problem correctly?
I confirm.
> What would happen if, for trixie only, bin:libc6 shipped two identical
> copies of ld-linux-x86-64.so.2, one in each of /lib64 and /usr/lib64?
That's an interesting idea. Do note that we don't actually have to ship
ld-linux in both locations. We can actually move it in a safe way
(unless we also move it between packages, which we don't). So let me
change that to: We keep /lib64 (the directory) in addition to
/usr/lib64. Keeping the directory prevents dpkg from deleting the
symlink (as it doesn't know about the filetype).
> Then at step 2, /lib64 does not get deleted and nothing breaks.
Confirmed (with the simplified variant).
> Later, whatever replaces /lib64 with a symlink needs to deal with this, but
> that's not significantly different than whatever it was going to do anyway,
> right? Just do this:
>
> 1. Whatever safety checks are appropriate.
> 2. Unless already verified to be identical by #1, hardlink
> /lib64/ld-linux-x86-64.so.2 to /usr/lib64/ld-linux-x86-64.so.2. This might
> be just a particular instance of the more general case of hardlink
> everything from /lib64 into /usr/lib64.
> 3. Unlink everything from /lib64.
> 4. Unlink /lib64.
> 5. Symlink /lib64 to /usr/lib64
I think we start from the premise that /lib64 already is a symlink and
as long as libc6 actually ships /lib64 (even if empty), dpkg won't
delete it. What we will not get here is getting rid of the aliasing and
we will also be unable to ship /lib64 as a symlink in any data.tar
(since that would be a directory vs symlink conflict, which has
unpack-order-dependent behaviour, which is bad).
> However, note that this cannot be a shell script, as then step 3 would
> delete /lib64/ld-linux-x86-64.so.2 and everything after that would fail.
Non-issue since we assume that bookworm is merged already.
> At that point, everything is fine, EXCEPT that dpkg now thinks it has a
> /lib64/ld-linux-x86-64.so.2 file installed, but really that is aliasing
> /usr/lib64/ld-linux-x86-64.so.2. When bin:lib6:amd64 is later upgraded (e.g.
> in forky) to a version that stops shipping /lib64/ld-linux-x86-64.so.2, dpkg
> will unlink /lib64/ld-linux-x86-64.so.2 and then everything breaks.
Simplified: dpkg thinks that it has /lib64 while it should not. When we
drop that in forky, stuff breaks.
> The fix to that is either whatever separate general case fix is being done
> for aliasing, or if the whole point is we are trying to avoid having that
> sort of thing at all, then just put in a special case that dpkg will not
> unlink /lib64/ld-linux-x86-64.so.2.
Yes, if we add that special casing to dpkg, we can remove /lib64 in forky.
> So we end up with something roughly like this in dpkg (please excuse
> syntax/pointer errors):
>
> Wherever file deletions are handled, make this change:
> - unlink(pathname);
> + special_unlink(pathname);
>
> to use this:
>
> char *SPECIAL_PATHS[] = {
> "/bin",
> "/lib",
> "/lib64",
> "/lib64/ld-linux-x86-64.so.2",
> "/sbin",
> NULL,
> }
>
> void special_unlink(const char *pathname) {
> const char **special;
> for (special = SPECIAL_PATHS ; *special ; special++) {
> if (strcmp(pathname, special) == 0) {
> return;
> }
> }
> unlink(pathname);
> }
Might work, but the list of SPECIAL_PATHS is /bin, /lib, /lib32,
/lib64, /libo32, /libx32, and /sbin and nothing else.
So yeah, I'm inclined to agree that this would technically work for
upgrades, but we'd not be closer to the bootstrap problem, because this
variant does not allow us to ship the symlinks in any data.tar for
trixie. So at the time of this writing, your this approach still looks
inferior to the variant I presented to me.
I'm definitely biased towards what I presented (cause I don't want to
make a fool of myself), so please continue pointing out issues and
alternative approaches. :)
Helmut
Reply to: