Re: DEP 17: Improve support for directory aliasing in dpkg
On Fri, 28 Apr 2023 at 09:09, Helmut Grohne <helmut@subdivi.de> wrote:
>
> On Thu, Apr 27, 2023 at 12:34:06AM +0200, Helmut Grohne wrote:
> > Ok, let's move on. I've proposed diversions as a cure, but in reality
> > diversions are a problem themselves. Consider that
> > cryptsetup-nuke-password diverts /lib/cryptsetup/askpass, which is
> > usually owned by cryptsetup. If cryptsetup were to move that file to
> > /usr, the diversion would not cover it anymore and the actual content of
> > askpass would depend on the unpack order. That's very bad and none of
> > what I proposed earlier is going to fix this.
> >
> > And of course, this is not some special example, it's a pattern:
> > * /lib/udev/rules.d/60-cdrom_id.rules: udev -> amazon-ec2-utils
> > * /sbin/dhclient: isc-dhcp-client -> isc-dhcp-client-ddns
> > * /bin/systemd-sysusers: systemd -> opensysusers
> > * ...
> >
> > So how do we fix diversions? Let's have a look into the dpkg toolbox
> > again. I've got an idea. Diversions. What you say? How do you fix
> > diversions with diversions? Quite obviously, you divert
> > /usr/bin/dpkg-divert! And whenever dpkg-divert is instructed to add a
> > diversion for a non-canonical path, you forward that call to the real
> > dpkg-divert, but also call it with a canonicalized version such that
> > both locations are covered. When initially deploying the diversion of
> > /usr/bin/dpkg-divert, we also need to transform existing diversions.
> > Other than that, things should work after doubling down on diversions.
> > Sorry, I don't have a test case for this yet.
>
> I still don't have a test case, but I have data. Using
> binarycontrol.d.n, I identified packages setting up diversions in
> preinst (this seems most common, but dash for instance sets up a
> diversion in postinst instead, so there are some false negatives). And
> while I initially tried to parse those preinst scripts, solving the
> halting problem seemed just too hard, so I opted for just running them.
> I'm attaching the relevant scripts and showing the affected diversions:
>
> diversion of /lib/udev/rules.d/60-cdrom_id.rules to /lib/udev/rules.d/60-cdrom_id.rules.disabled by amazon-ec2-utils in stable, testing, unstable
> diversion of /sbin/coldreboot to /lib/container/divert/coldreboot.orig by bfh-container in testing, unstable
> diversion of /sbin/halt to /lib/container/divert/halt.orig by bfh-container in testing, unstable
> diversion of /sbin/poweroff to /lib/container/divert/poweroff.orig by bfh-container in testing, unstable
> diversion of /sbin/reboot to /lib/container/divert/reboot.orig by bfh-container in testing, unstable
> diversion of /sbin/shutdown to /lib/container/divert/shutdown.orig by bfh-container in testing, unstable
> diversion of /lib/cryptsetup/askpass to /lib/cryptsetup/askpass.cryptsetup by cryptsetup-nuke-password in testing, unstable
> diversion of /sbin/dhclient to /sbin/dhclient-noddns by isc-dhcp-client-ddns in stable, testing, unstable
> diversion of /sbin/coldreboot to /lib/molly-guard/coldreboot by molly-guard in stable, testing, unstable
> diversion of /sbin/halt to /lib/molly-guard/halt by molly-guard in stable, testing, unstable
> diversion of /sbin/poweroff to /lib/molly-guard/poweroff by molly-guard in stable, testing, unstable
> diversion of /sbin/reboot to /lib/molly-guard/reboot by molly-guard in stable, testing, unstable
> diversion of /sbin/shutdown to /lib/molly-guard/shutdown by molly-guard in stable, testing, unstable
> diversion of /bin/systemd-sysusers to /bin/systemd-sysusers.real by opensysusers in stable, testing, unstable
> diversion of /sbin/coldreboot to /lib/open-infrastructure/container/divert/coldreboot.orig by progress-linux-container in stable, testing, unstable
> diversion of /sbin/halt to /lib/open-infrastructure/container/divert/halt.orig by progress-linux-container in stable, testing, unstable
> diversion of /sbin/poweroff to /lib/open-infrastructure/container/divert/poweroff.orig by progress-linux-container in stable, testing, unstable
> diversion of /sbin/reboot to /lib/open-infrastructure/container/divert/reboot.orig by progress-linux-container in stable, testing, unstable
> diversion of /sbin/shutdown to /lib/open-infrastructure/container/divert/shutdown.orig by progress-linux-container in stable, testing, unstable
> diversion of /bin/zcat to /bin/zcat.gzip by zutils in stable, testing, unstable
> diversion of /bin/zcmp to /bin/zcmp.gzip by zutils in stable, testing, unstable
> diversion of /bin/zdiff to /bin/zdiff.gzip by zutils in stable, testing, unstable
> diversion of /bin/zegrep to /bin/zegrep.gzip by zutils in stable, testing, unstable
> diversion of /bin/zfgrep to /bin/zfgrep.gzip by zutils in stable, testing, unstable
> diversion of /bin/zgrep to /bin/zgrep.gzip by zutils in stable, testing, unstable
>
> All other diversion affect /etc or /usr and I think we're not going to
> move any files from /usr to /. So this is a complete list as of today
> and I have to say, I expected it to be longer. In effect, we're talking
> about merely 8 packages.
>
> For completeness sake, I also looked at the other packages mentioning
> dpkg-divert in their preinst to catch false negatives. I'll skip
> diversions inside /usr as well as removals of diversions here:
> * amazon-ec2-net-utils: diversion inside /etc
> * angband: comment about diversions
> * arpwatch: comment about diversions
> * dash: complex use of conditional diversions via postinst
> * dist: comment about diversions
> * gpr: conditional diversion (inside /usr)
> * iputils-arping: check for an existing diversion
> * iputils-clockdiff: check for an existing diversion
> * iputils-ping: check for an existing diversion
> * ld10k1: comment about diversions
> * mailagent: comment about diversions
> * oping: checks for an existing diversion
> * psgml: comment about diversions
> * ucf: comment about diversions
> * wireshark-common: checks for an existing diversion
This is great data - indeed, 8 packages seems eminently fixable.
> So yeah, with the exception of dash, this looks fairly good. Let me also
> dive into dash. Unlike the majority of diverters, it diverts in postinst
> rather than preinst to allow controlling /bin/sh via debconf. A similar
> technique is in effect by gpr. In any case, this is special, because
> dash diverts its own files, so when moving dash's file, its diversions
> can be migrated at the same time. It merely means, that we cannot have
> debhelper just move files (as that would horribly break dash) and
> instead have to move files on a package-by-package way. We could also
> opt for removing dash's diversion in the default case and there even is
> a patch for doing so (#989632) since almost two years. Too bad we didn't
> apply it. In any case, as long as the file moving is not forced via
> debhelper, dash should be harmless.
If I understand correctly, by "forced via debhelper" you mean the
proposal of fixing the paths at build time, right? But if not via
that, it means having to fix all of them by hand, which is a lot - is
it possible to fix dash instead? Or else, we could add an opt-out via
one of the usual dh mechanisms, and use it only in dash perhaps?
> With this number, another option is on the table. Rather than divert
> dpkg-divert, we could just fix these 8 packages to duplicate their
> diversions for /usr and then when moving the underlying files add
> versioned Conflicts to the old version of diverters (none of which are
> essential). So this is an order of 15 uploads (8 diverters, 6 diverted
> packages, dash).
>
> Luca Boccassi kindly pointed me at config-package-dev though. This is a
> tool for generating local packages and it also employs dpkg-divert.
> There is a significant risk of breaking this use case. If we were to
> divert dpkg-divert and automatically duplicate diversions, this use case
> were automatically covered.
>
> I am unsure how to proceed here and request assistance from the
> debathena project to evaluate the situation. If possible, I'd like to
> avoid the complexity of wrapping dpkg-divert.
Which part of config-package-dev causes a conflict here? Is it
something that can be fixed? Given it's declarative, an upload + rdeps
rebuild should be all that's needed, assuming we know what the issue
is and how to fix it. As far as I can remember, it's a build-time
utility and everything it does is embedded in the target package's
maintainer scripts. But it's been a few years since I last used it, so
I might remember wrongly.
Kind regards,
Luca Boccassi
Reply to: