Re: Should .a library contains non-reallocatable code?
(I have reordered Bernard's mail and my responses to try to produce a
coherent whole.)
Bernhard R. Link writes ("Re: Should .a library contains non-reallocatable code?"):
> [examples not involving -Bsymbolic]
My view is that when trying to build a shared library that depends on
a static library, -Bsymbolic should always be used and the static
library should always be included in the shared library.
The fact that not using -Bsymbolic causes breakage is IMO expected.
The main thrust of your mail is arguments (A) against including the
static library in the .so which depends on it and (B) against using
-Bsymbolic. I am not convinced.
I still maintain that shared libraries depending on static libraries
should include a copy of the static library (rather than having
unresolved symbols and expecting the static library to be in the
executable). Consequently a static library which might have to be
used for this purpose must be built with -fPIC. (If there is a
corresponding shared library then this problem doesn't arise.)
Firstly, (A), whether to include a static library in a shared library
which uses it; or, alternatively, whether to expect executables
dynamically linked against the shared library to be also statically
linked against and hence contain the shared library:
> Unresolved symbols is just the way dependencies of libraries
> works. The only thing missing here is the missing dependency on the
> static library. But that is simply not possible with static
> libraries.
In practice, a .so built in the way you suggest (with unresolved
references to the dependent static libraries) is not sanely useable in
the ways people normally want to use shared libraries. See just below
for a detailed explanation:
> Ian Jackson <ijackson@chiark.greenend.org.uk> [150223 20:09]:
> > (In particular, this would vitiate libbar's stable API/ABI.)
>
> This you have to explain.
For libbar.so to be a reasonable thing to ship separately, it must
have a stable ABI. This is necessary so that programs
compile-time-linked against one version of libbar can be later
run-time-linked against a different version of libbar.
But if compile-time-linking against libbar.so requires statically
linking to a libfoo.a which does /not/ have a stable API, then any
executable which is built against a particular libbar.so contains
copies of parts of libfoo.a in the executable. When that executable
is then run-time-linked against a different version of libbar.so,
which was built against a different version of libfoo.a, things will
go wrong:
The code in the new libbar.so will contain references to symbols
(functions and data) in libfoo.a, but will have been compiled to
expect those functions and data to have the ABI in the new libfoo.a.
But the run-time linker will resolve those undefined symbol references
in the new libbar.so to the definitions provided by the copy of
libfoo.a in the executable, which have the ABI for the old libfoo.a.
So a .so built this way is not useful. In fact, it is worse than a
static library. At least with a static library the correct
corresponding versions of everything are assembled together at
compile-time-link, and the risk of ABI mismatches is much reduced.
So if (for a particular upper library) we can't make a .so which
includes the static lower library, then there should be no .so at all.
> If you think there is another way a dynamic library can use a static
> library please tell so.
I have done so. It involves -Bsymbolic. See my example.
(B) on whether to use -Bsymbolic.
> Note that this uses -Bsymbolic, by which you hide some issues and might
> introduce other problems (depending on your library).
This is FUD.
It's true that you need to be aware of what you're doing and that
there are still ways to screw up. But that does not mean that it is
impossible to do correctly. See below where I address those specific
issues you raise.
> There are only two possibilities:
>
> You want a static library, then the static library has to be linked into
> the executeable directly.
As my example demonstrates, this is not true. It is possible to link
the static library into a shared object.
Usually, there is no problem with two different copies of the static
library being contained within the process and both being in use.
The exceptions are:
(a) the ABI/API of the shared library mentions things in the API/ABI
of the static library. But in such a case the shared library itself
obviously does not have a stable ABI and making a shared library out
of it was probably a mistake.
(b) the static library expects exclusive use of process-global system
facilities (the most obvious examples being some uses of fork and
signals, especially in multithreaded programs). But in such a case
the shared library also demands (in the semantics of its API) such
exclusive use; so it is unreasonable to also link into the program,
and simultaneously use, a different shared library which also demands
such exclusive use (by virtue of its use of the the same underlying
static library). I think this is unreasonable even if in theory it
might be possible for the two exclusive users to be unified by
unifying the references to a single instance of the static librarary.
(Of course in fact this is not in practice possible without linking
the whole program statically.)
> You foo within a dynamic library. Then instead of creating a dynamic
> library that includes foo you can just as well create dynamic libfoo.so
> and link against that.
If a package does not provide a shared library for libfoo, that is
probably because libfoo does not have a stable ABI. In that case
creating libfoo.so is probably a mistake.
To respond to your specific counter-transcript:
> > See transcript below. In my example foo[12].[ch] represents a library
> > with an unstable API/ABI, provided as a static library only. Note
> > that it has symbols `x' and `foo' which behave differently in the
> > different versions.
...
> You still pollute the ABI with the details of the internals:
>
> if you try to change main.c to:
> #include <stdio.h>
> #include "foo1.h"
> #include "bar2.h"
> int main() {
> double rr1;
> foo(&rr1);
> int r1 = rr1;
> int r2 = bar2();
> printf("%d %d\n", r1, r2);
> return 0;
> }
>
> and run gcc -Wall main.c -L. -lbar2 -lfoo1 ; LD_LIBRARY_PATH=. ./a.out
> you get:
> foo2: 0x7f42db6f0a48
> foo2: 0x7f42db6f0a48
> 0 17
>
> (In this specific case a version script might helps, but I'd not bet
> on helping in all cases).
I don't know what exactly you ran. Your transcript is not complete
and I don't get the same results. I suspect you didn't pass
-Bsymbolic everywhere.
I changed main.c to contain the code you just showed, and reran by
./build script (same as before), and got this:
(64)ian@zealot:~$ ./build
+ egrep . bar.c bar1.c bar1.h bar2.c bar2.h foo.c foo1.c foo1.h foo2.c foo2.h main.c t.c build
[...]
main.c:#include <stdio.h>
main.c:#include "foo1.h"
main.c:#include "bar2.h"
main.c:int main() {
main.c: double rr1;
main.c: foo(&rr1);
main.c: int r1 = rr1;
main.c: int r2 = bar2();
main.c: printf("%d %d\n", r1, r2);
main.c: return 0;
main.c:}
[...]
+ gcc=gcc -Wall -Werror -Wmissing-prototypes -Wmissing-declarations
+ gcc -Wall -Werror -Wmissing-prototypes -Wmissing-declarations -c -fPIC -Wall foo1.c
+ ar rs libfoo1.a foo1.o
+ gcc -Wall -Werror -Wmissing-prototypes -Wmissing-declarations -shared -fPIC -Wl,-Bsymbolic bar1.c -L. -lfoo1 -o libbar1.so
+ gcc -Wall -Werror -Wmissing-prototypes -Wmissing-declarations -c -fPIC -Wall foo2.c
+ ar rs libfoo2.a foo2.o
+ gcc -Wall -Werror -Wmissing-prototypes -Wmissing-declarations -shared -fPIC -Wl,-Bsymbolic bar2.c -L. -lfoo2 -o libbar2.so
+ gcc -Wall main.c -L. -lbar1 -lbar2
+ LD_LIBRARY_PATH=. ./a.out
foo1: 0x7f283cb44a98
foo2: 0x7f283c943a98
17 17
(64)ian@zealot:~$
So it does work.
Remaining points from your mail:
> > Furthermore, as Simon Richter points out, this happens to work only
> > because 1. your foo.c happens not to contain any data references, and
...
> Data references work fine, too:
Either one of my points (1) or (2) is sufficient for your example to
work. But (1) is in general unlikely and (2) is undesirable (for the
reasons I have explained and will expand on below).
> > 2. your construction of bar.so (should be libbar.so) produces a shared
> > library with unresolved references to libfoo, which is not usual or
> > desirable.
>
> Using a static library is undesirable.
I don't understand how this is relevant in this context. The
desirability or otherwise of static libraries is not relevant when we
are asking *how* rather than *whether* we should create or use such a
library.
Thanks,
Ian.
Reply to: