Re: DEP 17: Improve support for directory aliasing in dpkg
On Thu, Apr 27, 2023 at 12:34:06AM +0200, Helmut Grohne wrote:
> Ok, let's move on. I've proposed diversions as a cure, but in reality
> diversions are a problem themselves. Consider that
> cryptsetup-nuke-password diverts /lib/cryptsetup/askpass, which is
> usually owned by cryptsetup. If cryptsetup were to move that file to
> /usr, the diversion would not cover it anymore and the actual content of
> askpass would depend on the unpack order. That's very bad and none of
> what I proposed earlier is going to fix this.
>
> And of course, this is not some special example, it's a pattern:
> * /lib/udev/rules.d/60-cdrom_id.rules: udev -> amazon-ec2-utils
> * /sbin/dhclient: isc-dhcp-client -> isc-dhcp-client-ddns
> * /bin/systemd-sysusers: systemd -> opensysusers
> * ...
>
> So how do we fix diversions? Let's have a look into the dpkg toolbox
> again. I've got an idea. Diversions. What you say? How do you fix
> diversions with diversions? Quite obviously, you divert
> /usr/bin/dpkg-divert! And whenever dpkg-divert is instructed to add a
> diversion for a non-canonical path, you forward that call to the real
> dpkg-divert, but also call it with a canonicalized version such that
> both locations are covered. When initially deploying the diversion of
> /usr/bin/dpkg-divert, we also need to transform existing diversions.
> Other than that, things should work after doubling down on diversions.
> Sorry, I don't have a test case for this yet.
I still don't have a test case, but I have data. Using
binarycontrol.d.n, I identified packages setting up diversions in
preinst (this seems most common, but dash for instance sets up a
diversion in postinst instead, so there are some false negatives). And
while I initially tried to parse those preinst scripts, solving the
halting problem seemed just too hard, so I opted for just running them.
I'm attaching the relevant scripts and showing the affected diversions:
diversion of /lib/udev/rules.d/60-cdrom_id.rules to /lib/udev/rules.d/60-cdrom_id.rules.disabled by amazon-ec2-utils in stable, testing, unstable
diversion of /sbin/coldreboot to /lib/container/divert/coldreboot.orig by bfh-container in testing, unstable
diversion of /sbin/halt to /lib/container/divert/halt.orig by bfh-container in testing, unstable
diversion of /sbin/poweroff to /lib/container/divert/poweroff.orig by bfh-container in testing, unstable
diversion of /sbin/reboot to /lib/container/divert/reboot.orig by bfh-container in testing, unstable
diversion of /sbin/shutdown to /lib/container/divert/shutdown.orig by bfh-container in testing, unstable
diversion of /lib/cryptsetup/askpass to /lib/cryptsetup/askpass.cryptsetup by cryptsetup-nuke-password in testing, unstable
diversion of /sbin/dhclient to /sbin/dhclient-noddns by isc-dhcp-client-ddns in stable, testing, unstable
diversion of /sbin/coldreboot to /lib/molly-guard/coldreboot by molly-guard in stable, testing, unstable
diversion of /sbin/halt to /lib/molly-guard/halt by molly-guard in stable, testing, unstable
diversion of /sbin/poweroff to /lib/molly-guard/poweroff by molly-guard in stable, testing, unstable
diversion of /sbin/reboot to /lib/molly-guard/reboot by molly-guard in stable, testing, unstable
diversion of /sbin/shutdown to /lib/molly-guard/shutdown by molly-guard in stable, testing, unstable
diversion of /bin/systemd-sysusers to /bin/systemd-sysusers.real by opensysusers in stable, testing, unstable
diversion of /sbin/coldreboot to /lib/open-infrastructure/container/divert/coldreboot.orig by progress-linux-container in stable, testing, unstable
diversion of /sbin/halt to /lib/open-infrastructure/container/divert/halt.orig by progress-linux-container in stable, testing, unstable
diversion of /sbin/poweroff to /lib/open-infrastructure/container/divert/poweroff.orig by progress-linux-container in stable, testing, unstable
diversion of /sbin/reboot to /lib/open-infrastructure/container/divert/reboot.orig by progress-linux-container in stable, testing, unstable
diversion of /sbin/shutdown to /lib/open-infrastructure/container/divert/shutdown.orig by progress-linux-container in stable, testing, unstable
diversion of /bin/zcat to /bin/zcat.gzip by zutils in stable, testing, unstable
diversion of /bin/zcmp to /bin/zcmp.gzip by zutils in stable, testing, unstable
diversion of /bin/zdiff to /bin/zdiff.gzip by zutils in stable, testing, unstable
diversion of /bin/zegrep to /bin/zegrep.gzip by zutils in stable, testing, unstable
diversion of /bin/zfgrep to /bin/zfgrep.gzip by zutils in stable, testing, unstable
diversion of /bin/zgrep to /bin/zgrep.gzip by zutils in stable, testing, unstable
All other diversion affect /etc or /usr and I think we're not going to
move any files from /usr to /. So this is a complete list as of today
and I have to say, I expected it to be longer. In effect, we're talking
about merely 8 packages.
For completeness sake, I also looked at the other packages mentioning
dpkg-divert in their preinst to catch false negatives. I'll skip
diversions inside /usr as well as removals of diversions here:
* amazon-ec2-net-utils: diversion inside /etc
* angband: comment about diversions
* arpwatch: comment about diversions
* dash: complex use of conditional diversions via postinst
* dist: comment about diversions
* gpr: conditional diversion (inside /usr)
* iputils-arping: check for an existing diversion
* iputils-clockdiff: check for an existing diversion
* iputils-ping: check for an existing diversion
* ld10k1: comment about diversions
* mailagent: comment about diversions
* oping: checks for an existing diversion
* psgml: comment about diversions
* ucf: comment about diversions
* wireshark-common: checks for an existing diversion
So yeah, with the exception of dash, this looks fairly good. Let me also
dive into dash. Unlike the majority of diverters, it diverts in postinst
rather than preinst to allow controlling /bin/sh via debconf. A similar
technique is in effect by gpr. In any case, this is special, because
dash diverts its own files, so when moving dash's file, its diversions
can be migrated at the same time. It merely means, that we cannot have
debhelper just move files (as that would horribly break dash) and
instead have to move files on a package-by-package way. We could also
opt for removing dash's diversion in the default case and there even is
a patch for doing so (#989632) since almost two years. Too bad we didn't
apply it. In any case, as long as the file moving is not forced via
debhelper, dash should be harmless.
With this number, another option is on the table. Rather than divert
dpkg-divert, we could just fix these 8 packages to duplicate their
diversions for /usr and then when moving the underlying files add
versioned Conflicts to the old version of diverters (none of which are
essential). So this is an order of 15 uploads (8 diverters, 6 diverted
packages, dash).
Luca Boccassi kindly pointed me at config-package-dev though. This is a
tool for generating local packages and it also employs dpkg-divert.
There is a significant risk of breaking this use case. If we were to
divert dpkg-divert and automatically duplicate diversions, this use case
were automatically covered.
I am unsure how to proceed here and request assistance from the
debathena project to evaluate the situation. If possible, I'd like to
avoid the complexity of wrapping dpkg-divert.
Helmut
Reply to: