Hello cruel world, Due to upstream ABI changes, it looks very likely that libfreetype is going to have to undergo a library transition in the near future[0]. The details are still being settled, and it's possible (though unlikely) that the library will be fixed so that no transition is needed, but in the meantime I'd like to leave you all with a public service announcement. There are currently 583 packages in unstable which depend on the libfreetype6 package. That means that if this transition happens today, it will be bigger than the KDE transition was; it will be bigger than the OpenSSL 0.9.8 transition was (469 binary packages in unstable depend on either libssl0.9.8 or libssl0.9.7 today). *And it shouldn't be*. The truth is that, although freetype is a widely used library (as the more or less standard handler in Debian for TrueType fonts), of those 583 packages in unstable which depend on libfreetype6, only *178* of them come from source packages which build-depend on libfreetype6-dev. That means that the vast majority of these packages are either missing a build-dependency, or, more likely, are ending up containing binaries that link against a library which they do not use directly. And freetype is not the only library like this in Debian. At DebConf 4, I gave a presentation [1][2] which discussed the issue of brittle library dependency trees in Debian, and the problems they cause over time for a distribution with a decentralized development model. For various reasons, this issue never made it to significant discussion on the mailing lists; this is now threatening to bite us big and bad with freetype, and it's time to raise awareness of this issue in order to try to get the freetype transition, and future transitions like it under control. I would encourage you to read the presentation in question, but I will also summarize here: due to accidents of history, the convention when linking an executable or shared library is to tell ld to recursively add references to *all* libraries in the dependency tree below it. This is wrong for modern GNU/Linux systems when using dynamic linking, because the dynamic linker recurses the dependency tree on its own; and following this convention means that executables get not the needed one reference to the library, but two or more. Where things *really* go wrong is when the library goes through an soname change. If two different versions of the library are specified in the ELF header, then either users can't install the package, or they can install it and then it segfaults. In either case, this situation causes a lot more work for maintainers, autobuilders, and the release team (trying to get packages into testing) than it should. This email, therefore, is a call-to-arms for maintainers to improve the library handling in their packages. The impetus for this request is the prospective freetype transition, but if you maintain *any* packages in C and C++, this mail is still addressed to you. At this point, many of you may be wondering why we don't just keep around old copies of the libraries in question through the transition, and save ourselves the trouble of fiddling with individual packages. It is actually my intention to keep around a libfreetype6 package through the transition due to the large number of reverse-dependencies, but this is only a partial solution, for two reasons: - there's still a cost to this, because someone has to maintain both copies of the lib - since these applications we're talking about depend on the lib without using it, almost all of them got the dependency because some lib they *do* use uses it, so loading two versions of the lib into memory at once as a result of a partial upgrade will almost always cause segfaults... and that's bad! Moreover, this is a problem that happens over and over again, every time there's a library transition. It's not hard to look around and see that over the course of a release cycle, we spend a lot of time managing library transitions -- time that could be better spent on enhancements that actually matter to our users. Pruning our package dependencies is (ideally) a one-time fix for each package, with long-term time savings for each maintainer (and for the release team!). Think how nice it will be to have only a third as many dependencies, and only have to rebuild your package for a third as many transitions! So what changes should you make to your packages to help? * Use Debian's libtool. One of the chief instigators of excess -l options in linker commands is libtool. In order to ensure that libtool works for static and dynamic linking, on both GNU/Linux and on other Unices, libtool keeps track of all dependencies of all the libs your package uses, and puts them on the linker line all at once. There's been discussion with libtool upstream about fixing this, but in the meantime, you can use the Debian version of libtool instead, which includes a patch to prune unused libs from the linker line. For details on how to update the libtool used in your package, please see Scott James Remnant's excellent howto at [3]. * Fix your pkg-config .pc files. If you are a library maintainer, *you* can help make library transitions easier by making sure you aren't contributing to the problem. The second biggest source of bogus lib dependencies is pkg-config. If you have a library package that provides a .pc file for use with pkg-config, please make sure that your upstream is using the Requires.private field. This field, added by Tollef Fog Heen in pkgconfig 0.18, makes it possible to separate out the libs everyone needs to link against for a package from the libs that are only needed for static linking. If your upstream hasn't adopted this field yet, please encourage them to do so! Since support for this field has been recently added, your package probably needs a build-dependency on pkg-config (>= 0.18) in order to use it. For example usage, please see /usr/lib/pkgconfig/cairo.pc in the libcairo2-dev package. * Don't use other *-config tools. While many libraries these days use pkg-config, there are also other libs which ship their own tools for querying library and header include paths, libs needed for linking, etc. The problem is that all of these tools I've seen are incapable of distinguishing between what's needed for static linking, and what's needed for dynamic linking. They're also almost all unnecessary when building Debian packages, because for any library that's following the FHS, the linker options needed are very simple: "-lfoo". So even though it may mean diverging from upstream (who can't depend on the FHS like we can), I would still recommend hard-coding the libraries you need instead of using complicated tools to pick up dependencies you don't. There are two common ways to hardcode such an override. The first would be to edit upstream's build scripts (e.g., Makefile/Makefile.in/Makefile.am, configure.ac, acinclude.m4...) to use a hard-coded value instead of a detected value. In general, I think this is a better option even though it means patching upstream code, because it means upstream changes to lib handling will be detected at patch time. The other way is to override a make variable in debian/rules. For an example of the first approach, please see [4]; for an example of the second, [5]. * If all else fails, use -Wl,--as-needed. If for some reason one part or another of the above doesn't work for your package, there is one more option available to you. Since binutils 2.15, it has been possible to pass -Wl,--as-needed as an option to gcc in order to tell ld to try to figure out which libraries are *really* needed. Opinions differ on whether this option is a good idea at all, and this option was broken on some architectures in the recent past, so it's good to avoid it if possible; and there are certainly some cases where it will do the wrong thing. But it's usually better than getting to rebuild your package six times a year for nothing! If you maintain a package that's affected by this issue, you can help today; there's no need to wait until your package is hit by a library transition to make the changes described above. Indeed, for packages which depend on libfreetype6, it's important that we begin fixing these as soon as possible to avoid another multi-month transition. A complete list of packages that are potentially affected by the freetype transition can be found at [6]. If you have any questions at all about this (or even objections, naturally), please ask on debian-devel. Cheers, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. vorlon@debian.org http://www.debian.org/ [0] http://bugs.debian.org/314385 [1] http://people.debian.org/~vorlon/dependencies.sxi [2] http://people.debian.org/~vorlon/dependency-hell/ [3] http://people.debian.org/~keybuk/libtool-updating.html [4] http://bugs.debian.org/340273 [5] http://bugs.debian.org/340163 [6] http://people.debian.org/~vorlon/freetype-without-builddeps.txt
Attachment:
signature.asc
Description: Digital signature