Re: DEP 17: Improve support for directory aliasing in dpkg

[Helmut Grohne <helmut@subdivi.de>]

Hi Luca,

On Fri, Apr 21, 2023 at 03:29:33PM +0100, Luca Boccassi wrote:
> After Bookworm ships I plan to propose a policy change to the CTTE and
> policy maintainers to forbid shipping files in the legacy directories
> altogether, followed by a debhelper change to adjust any stragglers
> automatically at build time and a mass rebuild, plus MBF for the small
> % that does not use dh and a piuparts test to stop migration for
> anything that is uploaded and doesn't comply. That should bring the
> matter to an end, without needing to modify dpkg.

I think we now learned that this is quite oversimplified, but possibly
fixable. At this time we know about the following problem areas:
 * file loss during upgrades
   + The main reason for having the moratorium
   + Possible workaround is using Conflicts
 * diversion mismatches
   + Possibly fixable by duplicating affected diversions
 * alternatives
   + Only become a problem if we try migrating them to canonical paths
 * triggers
   + Possible fix is duplicating trigger interest

This is problems we know about now, but it likely is not an exhaustive
list. This list was mostly guided by Guillem's intuition of what could
break at https://wiki.debian.org/Teams/Dpkg/MergedUsr and I have to say
that his intuition was quite precise thus far. Notably missing in the
investigation are statoverrides. However, we should also look for a more
generic approach that tries capturing unexpected breakage.

I noticed that the number of packages shipping non-canonical files is
relatively small. It's less than 2000 binary packages in unstable and
their total size is about 2GB. So I looked into binary-patching them and
attach the resulting scripts.
 * repackdeb.sh is a helper script for repacking an individual binary
   package and canonicalizing the contained paths.
 * autorepack.sh creates and apt repository of repacked .debs and calls
   repackdeb.sh repeatedly in that process. Should complete in half an
   hour.
 * createchroot.sh consumes the apt repository to create a chroot using
   mmdebstrap. In there, all paths should be canonical.
 * Depends: curl, devscripts, dpkg-dev, fakeroot, mmdebstrap, moreutils
 * You are expected to place these scripts in an empty directory and run
   them from there. Please read them before running them. You'll likely
   have to edit at least createchroot.sh to make it useful.

I'm not sure how to devise test cases from this yet.

 * I tried createchroot.sh with --include=zutils, because that's one of
   the packages where I expect breakage due to mismatching diversions.
   Sure enough, I got the expected unpack error from dpkg!

 * I tried createchroot.sh with --include=gnome to see it in action with
   a few more packages and that appeared successful. So in this scenario
   there were no unforeseen and readily visible problems.

If you end up performing tests using these or similar scripts, please
report your results. If you can think of relevant scenarios, please
tell.

Helmut