DEP17 /usr-move: most mitigations (M18) for aliased diversions (P3) are broken
Hello fellow developers,
I have bad news about /usr-move. The primary mitigation (M18) for
aliased diversions (P3) as implemented by me in quite a few packages is
broken. This was brought to my attention by independent reports of Emil
Södergren and Colin Watson on live-build and debian-installer-utils
respectively. I now have a better understanding of dpkg-divert (when it
is faced with aliasing) and need to redo most patches (probably about
20). Affected packages shall receive updated patches during January. I'm
sorry for the inconvenience caused.
If your have a lot of time and are interested in the underlying
technical details, I invite you to continue reading. Otherwise, consider
doing something more pleasant.
Fundamentally, I assumed that dpkg-divert --rename would only move a
file if it was considered installed according to the dpkg database. What
really happens is that --rename moves a file as long as it exists. In
many cases that I tested, this would happen to work by chance, but
important corner cases would be wrong. We start with an aliased file
being diverted. M18 says that the diversion should be duplicated to the
canonical location. Unfortunately, we cannot translate the diversion
target in the same way. Say we move /bin/foo to /usr/bin/foo in a
package upgrade. dpkg will first put all the new files into temporary
locations, then move them all into place (including /usr/bin/foo) and
then unlink any removed files (including /bin/foo). Due to a sanity
check, it notices that /bin/foo was just unpacked and skips its
deletion. This sanity check is not in effect when a file is diverted. If
we were to alias the diversion targets, the diverted file would be
reliably lost. This is why all of the duplicated diversions use a
.usr-is-merged suffix. This brings another consequence. When diverting,
we must rename the file depending on whether it is aliased or canonical,
but that's not what --rename does. No matter how we order the
dpkg-divert calls it will wrongly rename either the aliased or the
canonical case. And that's what we've seen in debian-installer-utils and
live-build.
As a result, I now propose using dpkg-divert with --no-rename for
duplicated diversions and doing the move manually consulting dpkg -S to
figure out where to move to. Beware that dpkg-divert is normally
idempotent and skips a rename when a file already is diverted, so as we
implement the rename externally, it must properly be conditional to the
diversion being newly added to retain idempotency.
Since I was failing to get this right earlier, I figured I could as well
write some tests. As a result, I'm attaching a shell script. It
constructs a few demo packages and attempts a hopefully exhaustive
combination of using them and dpkg-divert by writing its experiments to
the working directory. Running it should be fairly boring. If you happen
to review it, I appreciate a response.
The newly proposed logic becomes fairly elaborate. As there is no place
that could share it and we want to get rid of it fairly soon, I intend
to copy the code into the relevant users.
A slight simplification can be achieved if the diverting package depends
on a version of the diverted package that has canonicalized the
diverted file. Then we may use --rename on the canonical path and
--no-rename on the aliased path. As a result, adding the diversion in
preinst may wrongly rename the file to the canonical diversion target,
but the dependency ensures that it'll be clobbered with an updated
version. Once the dependency is unpacked, this error is papered over.
codesearching DEP17.*M18 (and a few other patterns) suggest the
following packages needing an update:
* bfh-metapackages
* clonezilla
* cryptsetup-nuke-password (and cryptsetup)
* debian-installer-utils
* isc-dhcp
* molly-guard
* live-build
* open-infrastructure-system-tools
* opensysusers
* piuparts
* progress-linux-metapackages
* systemd
* zutils (and gzip)
I've not reviewed them yet. debian-installer-utils has a fixed MR and
live-build definitely is broken. Please expect me to follow up on each
package within January.
Helmut
Reply to: