Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

[Ian Jackson <ijackson@chiark.greenend.org.uk>]

Russ Allbery writes ("Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> There's been a lot of discussion of this, but it seems to have been fairly
> inconclusive.  We need to decide what we're doing, if anything, for wheezy
> fairly soon, so I think we need to try to drive this discussion to some
> concrete conclusions.

Yes.

> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:

Thanks for this summary and analysis.  I agree with your conclusions.

> Does this seem comprehensive to everyone?  Am I missing any cases?

I think you have covered all of the cases that have been brought up on
this list, which I think are all of the important and frequent cases.

Thinking about other corner cases can be deferred for now because we
can put off converting affected packages until wheezy+1, or if we
really want to convert we can very probably add a "common" package.
So let us press on.

> * The binNMU process is changed to add the binNMU changelog entry to an
>   arch-qualified file (changelog.Debian.arch, probably).  We need to
>   figure out what this means if the package being binNMU'd has a
>   /usr/share/doc/<package> symlink to another package, though; it's not
>   obvious what to do here.

If we always put the binNMU changelog file in
 /usr/share/doc/<package>/changelog.Debian.<package>:<arch>
then in the symlink case we can put it file in 
 /usr/share/doc/<symlink-target>/changelog.Debian.<original-package>:<arch>
and everything will work (apart from the fact that some minority of
changelog-reading tools will need to be taught to look at the new path).

> Please note that this is a bunch of work.  I think the Lintian work is a
> good idea regardless, and it can start independently.  I think the same is
> true of the binNMU changelog work, since this will address some
> long-standing issues with changelog handling in some situations, including
> resolving just how we're supposed to handle /usr/share/doc symlinks.  But
> even with those aside, this is a lot of stuff that we need to agree on,
> and in some cases implement, in a fairly short timeline if this is going
> to make wheezy.

Yes.  The work that absolutely needs to be done ASAP seems to be:
 - put the refcounting back in dpkg
 - lintian support for arch-qualified overrides
 - update the binNMU machinery to write the new changelog file instead

Things that should be done but are not on the critical paths:
 - transpose the restrictions on use of refcounting into policy
   (for now they can go in a text file in dpkg-dev, or even just
   a reference to your email)
 - update changelog-reading tools to look for binNMU changelogs too

Things which we can do at our leisure:
 - convert individual libraries
 - think about whether to always arch-qualify the whole changelog
 - think other refcounting corner cases (see my comments above)

> 5. Data files that vary by architecture.  This includes big-endian
>    vs. little-endian issues.  These are simply incompatible with
>    multiarch as currently designed, and incompatible with the obvious
>    variations that I can think of, and will have to either be moved
>    into arch-qualified directories (with corresponding patches to the
>    paths from which the libraries load the data) or these packages
>    can't be made multiarch.

Yes.  Of these, arch-qualifying the path seem to be to be obviously
the right answer.  Of course eg if the data files just come in big-
and little-endian, you can qualify the path with only the endianness
and use refcounting to allow the equal-endianness packages to share.

Ian.