Re: ocaml compiled binaries and rpath
[As my knowledge of OCaml and rpath are very poor, I will give
my opinion about this issue ;)]
On Fri, 11 Apr 2003 18:26:01 +0200, Claudio Sacerdoti Coen wrote
[...]
> *** 5 ***
> So far, so good. But where are the -rpath? Let's go back to the
> libB directory and let's issue again the ocamlmklib command with the
> -verbose flag. This time, though, use the absolute path instead of
> ../installed. This is what we get:
>
> claudio@oberon:/tmp/rpath/libB$ ocamlmklib -o b -L/tmp/rpath/installed -la
> stub_b.o b.cmo b.cmx -verbose
> + gcc -shared -o dllb.so stub_b.o -L/tmp/rpath/installed
> -Wl,-rpath,/tmp/rpath/installed -la
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^
> + ar rc libb.a stub_b.o
> + ranlib libb.a
> + /usr/bin/ocamlc -a -o b.cma b.cmo -dllib -lb -cclib -lb -ccopt
> -L/tmp/rpath/installed -ccopt -Wl,-rpath,/tmp/rpath/installed -cclib -la
> + /usr/bin/ocamlopt -a -o b.cmxa b.cmx -cclib -lb -ccopt
> -L/tmp/rpath/installed -ccopt -Wl,-rpath,/tmp/rpath/installed -cclib -la
>
> Xavier decided to add a -rpath option (it does so every time the -L is
> followed by an absolute path). What is the gain? Type "make install"
> and let's go back to testB. Unset LD_LIBRARY_PATH and try make again.
> THIS TIME IT WORKS! The -rpath tells the linker where the library is
> to be found! Thus we do not need to use LD_LIBRARY_PATH in any way.
> [ That's good! ]
>
> Still, let's try to execute testB: it does not work!!! The -rpath is
> used only by the linker, not by the loader!
Hey this is normal, -rpath is only used when compiling libB, so testB
cannot guess where dllb is living, this has nothing to do with -rpath.
If you compile testB with
ocamlc -I ../libB/ -dllpath /tmp/rpath/installed -dllib dllb -o testB testB.ml
then it works :)
This means that dllb finds dlla at run-time.
> **************************
>
> To sum up: Xavier decided to use the -rpath trick to avoid setting
> LD_LIBRARY_PATH when linking.
I disagree, rpath stands for 'runtime library search path', according to ld(1).
In the section about the -rpath-link option, there are detailed
explanations about search paths, and indeed LD_LIBRARY_PATH can be used,
but LD_RUN_PATH should be used instead. The libtool info documentation
tells that when finishing library installation,
You will also need to do one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable
during linking
- use the `-RLIBDIR' linker flag
Hmmm, LD_LIBRARY_PATH is intended for execution whereas LD_RUN_PATH is
for linking, how do they want us to understand this issue?
So the -rpath trick can indeed avoid setting LD_LIBRARY_PATH when linking,
but its main goal is to hardcode library location to ensure it cannot be
overridden, see http://www.visi.com/~barr/ldpath.html for some
explanations.
Alexandre Oliva debated the -rpath issue to death with Debian folks when
we migrate from libc5 to libc6; he made valuable statements in
http://lists.debian.org/debian-devel/1999/debian-devel-199901/msg02260.html
IMO he was basically right, but there are other corner cases which have not
been discussed (or maybe I missed them since I did not read the whole
thread ;)).
Let's have a closer look at how Debian migrates from libc5 to libc6.
>From what I read, -rpath was disabled when compiling, libc5 libraries
were moved into /usr/lib/libc5-compat and new libc6 libraries
into /usr/lib, The compat library was added into /etc/ld.so.conf so
that old binaries continue to work, but there are trouble when the
same libraries are built against libc5 and libc6, which was prevented
by making use of Conflicts fields.
You will find many links similar to
http://people.debian.org/~che/personal/rpath-considered-harmful
but remember Alexandre's argument, not bumping sonames was fucking
stupid, so all this machinery was certainly not a good idea.
This was for the libc migration.
AFAICT OCaml situation is very different:
* library location is not well established
* there are no sonames (is it true?), and this makes a huge difference
> Instead he hard-coded the stublibs directory in the produced code,
> so that the loader does not complain.
> One trick is for the linker, the other one for the loader.
> The second trick works only for the stublibs directory. The first one
> always work (but makes lintian complain). Moreover there is also the
> symbolic link trick from dllname.so to libname.so that should be done
> by hand.
Now back to your example. When packaging programs, we usually
a. build libraries
b. build binaries against these libraries
c. test binaries
d. install libraries and binaries if everything worked fine
You showed how to use -rpath to perform these operations, but remember
that as package managers we do not install files to their definitive
location, but to a staging area.
So let's try something else; imagine that your staging area is /tmp,
then you compile with -rpath=/rpath/installed. In order to prevent
linkage and execution failures, you set LD_LIBRARY_PATH to
/tmp/rpath/installed, and it works! Great.
But there is one major problem, imagine that you installed everything,
and 3 months later there is a new release. You compile as usual,
and it seems to work, but in fact you run test binaries against
*old* installed libraries, because rpath is searched before LD_LIBRARY_PATH
and it contains the requested files. It is really painful, because
those errors are very hard to diagnose.
I do not know of any viable solution to build packages because of
this problem, old libraries must be removed before building.
One might propose to Build-Conflicts against previous library versions,
but is it really what we want? ;)
> What happens if you remove the -rpath from a debian package? You may
> break the compilation processes of other people that are expecting
> your library to have the -rpath and that do not have the LD_LIBRARY_PATH
> set to stublibs (or other strange directories).
I do not follow you. OCaml is configured so that libraries are found in
the stublibs directory, so there is no need for LD_LIBRARY_PATH, and this
is good.
The -rpath is useful if you have 2 incompatible versions of the same
library in different locations, and you want to link against a specific
one.
AFAICT this does only make sense with 3rd party softwares, I cannot find
any example where -rpath is useful for Debian packages.
As stated above it is harmful because you cannot test programs when
building them. Moving libraries is also a bit tricky, you have to
set LD_LIBRARY_PATH if you want your programs to run.
> Final note: of course the problem solved by rpath was much more critical
> _before_ the introduction of the single stublib directory (those times
> when my LD_LIBRARY_PATH used to be several lines long).
>
>
> I hope that all this could be useful to somebody: my wirst is aching!
Sure, your example was really great, it helped me a lot to understand
how to compile OCaml libraries.
Denis
Reply to: