On Tue, 2007-12-25 at 20:12 -0800, Steve Langasek wrote: > On Fri, Dec 21, 2007 at 10:55:32AM +0000, Neil Williams wrote: > > On Thu, 20 Dec 2007 21:44:19 -0800 > > Steve Langasek <vorlon@debian.org> wrote: > > > But *nothing* here guarantees that the version of libbar the application is > > > linked against has the same ABI as the one libfoo itself linked against! > > > ... until both the application and libfoo are rebuilt. So the issue > > here is triggering rebuilds of reverse dependencies of libbar? > > No. That doesn't cause previously released binaries to blink out of > existence. > > What's required is to ensure that there's a package rename each time > libfoo's ABI changes, including when the libfoo ABI change is caused by type > changes in the underlying lib. This is essentially the general case of the > c102, c2, c2a, and ldbl transitions that we've been through for compiler > changes over the past few years; in theory we should be able to accomplish > these transitions for libraries with fewer reverse-dependencies than > libstdc++ with significantly less pain, as long as library maintainers are > forewarned. > > But once the package name change is handled, the rebuild of reverse-deps > follows naturally. That sounds a lot better than the original idea of incorporating the entire dependency chain into the package name. I think I'm doing this already - if libfoo1 implements and exports types from libbar2 and libbar2 moves to libbar3, I would expect to have to port libfoo to libbar3 and this would usually cause a SONAME bump in libfoo. So could I ask, from an upstream perspective, what kind of changes in the underlying lib might *not* cause such a port and therefore end up with libfoo1 still being buildable against libbar3 yet *still* require a SONAME bump to accommodate the transition? Have I got these possible scenarios correct: 1. libfoo depends on libbar2 without exporting libbar symbols. libfoo can migrate to libbar3 with internal code changes and if these do not require changing any libfoo interfaces, libfoo needs no SONAME bump. pkgconfig does not export libbar in the libfoo --libs data. If an application uses libbar and libfoo (for whatever reason), the ported version of that application that uses libbar3 simply needs to depend on the version of libfoo that depends on libbar3. In this case, an automated rebuild of the application could miss this step, causing breakage, yet forcing a change in the libfoo package name seems unnecessary (especially if there are other applications that depend on libfoo but not directly on libbar). Another problem is if the application ports to libbar3 before libfoo - currently, there is no way of reliably picking this up except the application maintainer being particularly careful with the pbuilder logs and noticing that libbar2 is still being installed during the build as well as libbar3 due to the indirect dependency on libfoo. Maybe pbuilder could highlight such incidences? (or even fail the build if it happens)? Maybe sbuild too? However, this would again raise the spectre of being unable to fix critical bugs in the application should libfoo be slow to port to libbar3. Again, it's a balance of risk - problems are *possible* when the app is linked against libbar3 and indirectly against libbar2 but in the absence of real bugs, is it worth causing more delays to other bug fixes? Does libfoo need to use versioned symbols to cope with this or is it simply down to those applications that use libfoo and libbar to specify the right version of libfoo once the application itself has ported to libbar3? (And wait until libfoo has also migrated?) What methods are needed to ensure that the application does this? (e.g. if left without changes, an application directly dependent upon libbar3 and libfoo could be built for unstable and presumably migrate into testing before the updated libfoo). There needs to be some check on the application that compares the dependency chain as well as the dependency list in the package itself. lintian cannot do that. 2. libfoo depends on libbar2 and exports some libbar symbols in libfoo-dev. libfoo can migrate to libbar3 with only internal code changes because it only exports a portion of the libbar API that has not actually been changed in the libbar2->libbar3 transition. libfoo needs to export libbar in the libfoo --libs data. Is a SONAME bump needed in libfoo even though *none* of the libfoo interfaces have changed? As upstream for libfoo, I would not normally make a SONAME bump in this situation. Would it be better to rename the Debian package without bumping the SONAME, e.g. libfoo1-0->libfoo1-1 ? Wouldn't this cause unnecessary rebuilds of applications that only depend on libfoo and not libbar? 3. libfoo depends on libbar2 and needs to make changes to its own interfaces to cope with the libbar transition or needs to export symbols from the new libbar API and therefore makes a SONAME bump. No problem here except ensuring that applications transition to both API's at the same time. Can be enforced by making libbar-dev conflict with the old libfoo-dev or making the new libfoo-dev depend on the new libbar-dev version. It is only in case 2: that I find confusion and a need for some kind of Debian-only package name change in libfoo. Is that correct? Now it may be that for case 2: to happen in real situations, libbar would actually turn out to be a collection of libraries: libbar-x11.so and libbar-sql.so. Say the libbar2->3 transition only affects libbar-x11 and it is libbar-x11 that is NEEDED by the application, not libbar-sql: Old package: libbar-x11.so.2.1.3 libbar-sql.so.2.4.1 New package: libbar-x11.so.3.0.0 libbar-sql.so.2.5.0 A package split in libbar would actually seem to be the better solution, yes? Have the libbar source build an extra set of packages: libbar-x113, libbar-x11-dev, libbar-x11-dbg, libbar-doc libbar-sql2, libbar-sql-dev, libbar-sql-dbg libbar-dev as a meta package depending on libbar-sql-dev and libbar-x11-dev pkgconfig data for libbar-x11 in bar-x11.pc pkgconfig data for libbar-sql in bar-sql.pc Remove bar.pc or fold bar-x11.pc and bar-sql.pc into a new bar.pc to support those applications that use both. Get the whole thing done in one spell in NEW. To cope with this transition, libfoo could simply swap the pkg-config call from: pkg-config --libs bar to pkg-config --libs bar-sql No SONAME bump, just bump the version of the libbar-dev dependency (or preferably replace libbar-dev with libbar-sql-dev). Would this be sufficiently beneficial that the libbar maintainer would be considered remiss for not implementing such a split? > > > If > > > libfoo linked against libbar1, the pkg-config approach only ensures that > > > when your application is built against the libbar2 version of libbar-dev, > > > the resulting binary will depend on both libbar1 and libbar2 (despite not > > > using any symbols from libbar2). All this does is increase the chances of > > > segfaults or bad runtime behavior as a result of symbol clobbering. > > > Maybe I've got this wrong but there would appear to be a few methods to > > fix this: > > 1. Incorporating the libbar SONAME into the libfoo package SONAME - as > > Simon Richter recommended - this could quickly end up with > > libany1-foo2-bar2-baz0-base4-pango0-... etc and I'm not convinced that > > this would actually help anyone if, for example, libbaz0 migrates to > > libbaz1 in a way that completely breaks libbar2. > > Then either libbar2 becomes libbar3 at the same time, or libbar becomes > unsupported (and the stack above it has to be ported or else it's not > releasable), or libbaz needs to support symbol versioning to permit > coexistence of the two versions in the same address space. Even if the actual fix for libbar2 is only internal? Say a change in libbaz meant that the core functionality of libbar fails. As "core functionality", it would be used by libbar but not exported so libbar would not export libbaz symbols; there is no need for libbar2 to become libbar3. Yet the problem isn't in libbaz either, so why force libbaz to implement versioned symbols through (apparently) no fault of its own? To me, this whole problem comes down to the applications, not the shared libraries or pkgconfig. Problems only arise if one package is linked directly and indirectly against different versions of the same object. The problem should be solved at the point at which it arises - in whichever package ends up with the double linkage. Changing all the shared libraries names and causing lots more transitions with unnecessary SONAME bumps seems the wrong approach, IMHO, because we end up with more transitions, not easier transitions. Lintian cannot check beyond the confines of the .changes file for one source package so we need a supplementary check and one place for this would appear to be all those scripts that build in a chroot. What about making the version of libbar-dev that depends on libbar3, conflict with libbar2? The application maintainer would soon discover the problem because the application dependency on libfoo could not be satisfied in a pbuilder environment until libfoo updates to libbar3 itself. This gives time for the application maintainer to consider an interim release of the current (unported) version of the application dependent on libfoo and libbar2, if it is possible to fix certain bugs that way, and seek an upgrade of libfoo in the meantime. i.e. leave pkgconfig out of it entirely. This has the advantage that none of the applications that only depend on libfoo (and libbar indirectly) would be affected - it is limited to those who need both libfoo-dev (which brings in libfoo and libbar2) and libbar-dev (which brings in libbar3 and complains about libbar2). It can also be implemented entirely within the one library that is undergoing the transition: libbar. It also allows applications that still use libbar2 to coexist with libbar3 - only when trying to build against libbar3 does it become a problem. The biggest problem with that approach is upstream. It complicates porting the application itself - depending on how many other packages would be removed by apt when trying to install libbar-dev on a real system. There would be no point making libgtk+-3.0-dev conflict against libgtk-2.0-0. However, I think this is manageable for small transitions and less common libraries - the very ones that would tend to get missed by the current methods. Everyone knows when a gtk transition is taking place, everyone is watching for situations where a package gets linked against libgtk+2.0.so and libgtk+3.0.so and there is little chance that any program would work with such duplication. Maybe a check that can be confined to chroots is needed - a function within pbuilder that checks the build dependencies being requested and fails if libbar2 and libbar3 are being installed at the same time during a build? (The mere fact both are needed in a pbuilder build should be evidence enough that at least one object in the dependency list is going to end up linked to both.) It would seem relatively simple to implement: sort the fully parsed dependency list before calling 'apt-get' to remove duplicates, strip the SONAME from the list of dependencies, error if any of the remaining strings are not unique. Something like (although probably not needing tmpfiles): #!/bin/sh -e LIST="libfoo1 libbar2 libbar3 myprog libfoo1" rm -f tmpfile for PKG in $LIST; do echo $PKG >> tmpfile done ULIST=`sort -u < tmpfile` for PKG in $ULIST; do PKG=`echo "$PKG" | sed -e 's/[0-9]$//'` echo $PKG >> tmpfile2 done ORIG=`grep -c '[a-z]' tmpfile` SORT=`grep -c '[a-z]' tmpfile2` if [ $ORIG -ne $SORT ]; then echo -n "FAIL: " echo $ULIST | sed -e 's/\n//' exit 1 fi echo "OK" gives: FAIL: libbar2 libbar3 libfoo1 myprog I'm sure that can be improved to only show the problematic packages. -- Neil Williams ============= http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/
Attachment:
signature.asc
Description: This is a digitally signed message part