Re: dpkg-shlibdeps couldn't find library ... warning vs error
Hi!
[ Just rereading my reply, I should stop sending mail with a sleep deprived
brain, sorry! :) ]
On Fri, 2019-06-28 at 08:09:11 -0400, Crim, Christopher wrote:
> On Thursday, June 27, 2019 9:09:34 PM EDT Guillem Jover wrote:
> > On Wed, 2019-06-26 at 14:58:34 -0400, Crim, Christopher wrote:
> > > I am trying to understand the regex rules in the function split_soname
> > > from
> > > dkpg-shlibdeps.pl. Working with an old corporate code base that has yet
> > > to
> > > adopt any versioning, all of the shared objects are named <object>.so.
> > > When attempting to package on a box that does not yet have the dependent
> > > packages installed dpkg-shlibdeps will throw warnings, "dpkg-shlibdeps:
> > > warning: couldn't find library <lib>.so needed by ..." but packaging will
> > > still succeed. We have few shared objects that have ".so." as part of
> Just to be clear, an example name would be "libcom.foo.so.system.utility.so".
> The reason we had to add the domain prefix was that we were running into name
> collisions. "libcom.bar.ss.pos.utility.so" would have conflicted with the
> above without the domains.
I think this is a pretty uncommon practice. I'd probably recommend
using «-» instead of «.» to avoid possible matches if there's a «.so.»
inbetween. And to add a version too.
> > > their name. When dpkg- shlibdeps encounters these dependencies the
> > > warning becomes an error, "dpkg- shlibdeps: error: couldn't find library
> > > <lib>.so needed by ..." and packaging fails. I traced this down to the
> > > split_soname function which checks 2 regex comparisons. The shared
> > > objects that only throw warnings fall into the "else" clause of the
> > > function. The ones that fail fall into the first if clause which as
> > > regex 1. What were these regexs meant to match?
> >
> > If the installed packages have no shlibs or symbols file information
> > then dpkg-shlibdeps cannot know what dependency information to inject,
> > which means the generated binaries will end up with missing/incorrect
> > dependencies. See below for the distinction.
> >
> > > regex 1 (/^(.*)\.so\.(.*)$/):
> > > was this meant to capture shared objects with so number and revisons such
> > > as libc.6 or libbz2.so.1.0.4? If so, should the values after "so" be
> > > restricted to numbers?
> >
> > It cannot be restricted to numbers only because the version might
> > include other characters.
>
> I agree now, I did find some examples on my system:
> libtidy.so.5deb1.6.0
> libKPimSMTP.so.5.9.3.abi1
Right.
> > This format is the one used commonly for public and stable shared
> > libraries, where the format is <name>.so.<version>, and then there's
> > at least one symlink of the form <name>.so pointing to the former and
> > used by other objects to link to, that gets them the latest <version>.
This part is correct.
> > The key here is that the SONAME might only include part of the
> > SOVERSION, depending on where the <name>.so symlink points to, so you
> > could have:
> >
> > libfoo.so.1.2.3
> > libfoo.so.1
> > libfoo.so → libfoo.1
This is very confused. The unversioned symlink (or what it resolves to)
does not need to have a direct relation to the SONAME, which is what gets
encoded in the shared library at link time usually via a command-line
argument.
So for example (following the one you queried about):
$ realpath /lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libc-2.28.so
$ objdump -p /lib/x86_64-linux-gnu/libc.so.6 | grep SONAME
SONAME libc.so.6
In this case libc uses (for historical reasons) a filename encoding
the project version name (which is distinct to its SOVERSION), but its
SONAME encodes the arch-specific SOVERSION. The actual requirement
here is that the unversioned «<name>.so» used during compile/link-time
refers to the version intended to be linked against, and that the
SONAME can then be found at run-time, directly or via symlinks.
> > Because these are public interfaces they are expected to have
> > dependency information available.
More or less I guess.
> > > regex 2 (/^(.*)-(\d.*)\.so$/)
> > > The terminating so on this regex helps limit its capture but again should
> > > the groups be limited to digits only?
> > This is the format commonly used for private and/or unstable shared
> > libraries, where the whole version is encoded in the SONAME. So you
> > could have:
I think this is also correct in general.
> I was not aware of this distinction. A quick search did not point me to
> anything stating that. Does this mean that libc on my machine is unstable?
> (libc-2.28.so)
> > libbar-1.2.3.so
> > libbar.so → libbar-1.2.3.so (optionally, otherwise you might be
> > required to specify the whole SONAME)
An example of this could be:
$ realpath /usr/lib/x86_64-linux-gnu/libbfd-2.31.1-system.so
/usr/lib/x86_64-linux-gnu/libbfd-2.31.1-system.so
$ objdump -p /usr/lib/x86_64-linux-gnu/libbfd-2.31.1-system.so | grep SONAME
SONAME libbfd-2.31.1-system.so
In this case there's no unversioned «<name>.so» symlink, because this
is a private interface, so projects need to know the exact version to
link to.
> > As these are private/unstable interfaces they are to be used at your
> > own risk, so that's why they are just warnings.
This is partial non-sense, sorry. :) The first part holds, but not the
second.
> These do not throw warnings though, they throw errors. At link time, if the
> library depended on had an soname that name would be put in the NEEDED field.
> If the library depended on did not have an soname or any version info then the
> just the name would be in the NEEDED field.
> libfoo.so.1 will fail
> libbar-1.2.3.so will fail
> libbaz.so will just be a warning
Yes, sorry. The recognized formats throw errors, the unknown ones
warnings. It is expected that both recognized formats provide
dependency information in the form of shlibs or symbols files.
> So it can't be that dpkg-shlibdeps only wants to throw warnings for private/
> unstable libraries and errors for everything else. I guess this gets to the
> heart of my question, why is there a distinction. If it cannot find a
> dependency should it just always fail?
Yes, sorry for the confusion. :)
> > I'm not sure now, there's anywhere properly documenting these details.
> > I don't think the debian-policy manual contains rationale for these.
> > I've for now locallt documented the split_soname() function, willcheck
> > whether one of the man pages can also be improved.
(Some heavy typos in there :)
Thanks,
Guillem
Reply to: