Re: -Wl,--as-needed considered possibly harmful

[Steve Langasek <vorlon@debian.org>]

On Wed, Dec 26, 2007 at 01:11:40PM +0000, Neil Williams wrote:
> > > ... until both the application and libfoo are rebuilt. So the issue
> > > here is triggering rebuilds of reverse dependencies of libbar?

> > No.  That doesn't cause previously released binaries to blink out of
> > existence.

> > What's required is to ensure that there's a package rename each time
> > libfoo's ABI changes, including when the libfoo ABI change is caused by type
> > changes in the underlying lib.  This is essentially the general case of the
> > c102, c2, c2a, and ldbl transitions that we've been through for compiler
> > changes over the past few years; in theory we should be able to accomplish
> > these transitions for libraries with fewer reverse-dependencies than
> > libstdc++ with significantly less pain, as long as library maintainers are
> > forewarned.

> > But once the package name change is handled, the rebuild of reverse-deps
> > follows naturally.

> That sounds a lot better than the original idea of incorporating the
> entire dependency chain into the package name. 

> I think I'm doing this already - if libfoo1 implements and exports types
> from libbar2 and libbar2 moves to libbar3, I would expect to have to
> port libfoo to libbar3 and this would usually cause a SONAME bump in
> libfoo.

> So could I ask, from an upstream perspective, what kind of changes in
> the underlying lib might *not* cause such a port and therefore end up
> with libfoo1 still being buildable against libbar3 yet *still* require a
> SONAME bump to accommodate the transition?

Succinctly addressed by Robert.

> Have I got these possible scenarios correct:

> 1. libfoo depends on libbar2 without exporting libbar symbols. libfoo
> can migrate to libbar3 with internal code changes and if these do not
> require changing any libfoo interfaces, libfoo needs no SONAME bump.

No, "exporting libbar symbols" is not the right line to draw.  The right
line is the one I already identified, namely, "libfoo uses some of libbar's
types in its own exported ABI".

> Does libfoo need to use versioned symbols to cope with this

No, libbar does, as the library which has other libraries for
reverse-dependencies.

> There needs to be some check on the application that compares the
> dependency chain as well as the dependency list in the package itself.
> lintian cannot do that.

Not if libbar uses symbol versioning.

> 2. libfoo depends on libbar2 and exports some libbar symbols in
> libfoo-dev. libfoo can migrate to libbar3 with only internal code
> changes because it only exports a portion of the libbar API that has not
> actually been changed in the libbar2->libbar3 transition. libfoo needs
> to export libbar in the libfoo --libs data. Is a SONAME bump needed in
> libfoo even though *none* of the libfoo interfaces have changed? As
> upstream for libfoo, I would not normally make a SONAME bump in this
> situation. Would it be better to rename the Debian package without
> bumping the SONAME, e.g. libfoo1-0->libfoo1-1 ? Wouldn't this cause
> unnecessary rebuilds of applications that only depend on libfoo and not
> libbar?

If libfoo's exported ABI hasn't changed, then any package name or soname
change is gratuitous and should be avoided.

> 3. libfoo depends on libbar2 and needs to make changes to its own
> interfaces to cope with the libbar transition or needs to export symbols
> from the new libbar API and therefore makes a SONAME bump. No problem
> here except ensuring that applications transition to both API's at the
> same time. Can be enforced by making libbar-dev conflict with the old
> libfoo-dev or making the new libfoo-dev depend on the new libbar-dev
> version.

Requires disruptive name changes to libbar-dev which affect all packages
that build-depend on it, and is therefore again inferior to implementing
symbol versions up front.

> > > Maybe I've got this wrong but there would appear to be a few methods to
> > > fix this:
> > > 1. Incorporating the libbar SONAME into the libfoo package SONAME - as
> > > Simon Richter recommended - this could quickly end up with
> > > libany1-foo2-bar2-baz0-base4-pango0-... etc and I'm not convinced that
> > > this would actually help anyone if, for example, libbaz0 migrates to
> > > libbaz1 in a way that completely breaks libbar2.

> > Then either libbar2 becomes libbar3 at the same time, or libbar becomes
> > unsupported (and the stack above it has to be ported or else it's not
> > releasable), or libbaz needs to support symbol versioning to permit
> > coexistence of the two versions in the same address space.

> Even if the actual fix for libbar2 is only internal?

Then that would eliminate the first option, leaving only the other two.

> Yet the problem isn't in libbaz either, so why force libbaz to implement
> versioned symbols through (apparently) no fault of its own?

Because *all* libraries which have other libs as reverse-dependencies should
implement symbol versioning, precisely as the shortest path for reliably and
permanently addressing the various issues we're discussing.

It *is* libbaz's fault for having an ABI-breaking change.  And based on
history, it is only reasonable to assume all such libraries are guilty, and
demand that they use symbol versioning in anticipation of such future ABI
changes.

> To me, this whole problem comes down to the applications, not the shared
> libraries or pkgconfig. Problems only arise if one package is linked
> directly and indirectly against different versions of the same object.
> The problem should be solved at the point at which it arises - in
> whichever package ends up with the double linkage.

No.  You seem to agree that each soname change of a library near the bottom
of the stack should not propagate its way up the stack to affect other
libraries that themselves are not affected by the ABI change, but you again
overlook (or disdain) the impact that this has on partial upgrades if we
don't require those base libraries to use symbol versioning.  Neither the
rebuild, nor the installation, of packages against the new version of libbar
is instantaneous.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org