[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: dpkg-shlibdeps couldn't find library ... warning vs error


[ Just rereading my reply, I should stop sending mail with a sleep deprived
  brain, sorry! :) ]

On Fri, 2019-06-28 at 08:09:11 -0400, Crim, Christopher wrote:
> On Thursday, June 27, 2019 9:09:34 PM EDT Guillem Jover wrote:
> > On Wed, 2019-06-26 at 14:58:34 -0400, Crim, Christopher wrote:
> > > I am trying to understand the regex rules in the function split_soname
> > > from
> > > dkpg-shlibdeps.pl.  Working with an old corporate code base that has yet
> > > to
> > > adopt any versioning, all of the shared objects are named <object>.so. 
> > > When attempting to package on a box that does not yet have the dependent
> > > packages installed dpkg-shlibdeps will throw warnings, "dpkg-shlibdeps:
> > > warning: couldn't find library <lib>.so needed by ..." but packaging will
> > > still succeed. We have few shared objects that have ".so." as part of

> Just to be clear, an example name would be "libcom.foo.so.system.utility.so".  
> The reason we had to add the domain prefix was that we were running into name 
> collisions.  "libcom.bar.ss.pos.utility.so" would have conflicted with the 
> above without the domains.

I think this is a pretty uncommon practice. I'd probably recommend
using «-» instead of «.» to avoid possible matches if there's a «.so.»
inbetween. And to add a version too.

> > > their name.  When dpkg- shlibdeps encounters these dependencies the
> > > warning becomes an error, "dpkg- shlibdeps: error: couldn't find library
> > > <lib>.so needed by ..." and packaging fails.  I traced this down to the
> > > split_soname function which checks 2 regex comparisons.  The shared
> > > objects that only throw warnings fall into the "else" clause of the
> > > function.  The ones that fail fall into the first if clause which as
> > > regex 1.  What were these regexs meant to match?
> > 
> > If the installed packages have no shlibs or symbols file information
> > then dpkg-shlibdeps cannot know what dependency information to inject,
> > which means the generated binaries will end up with missing/incorrect
> > dependencies. See below for the distinction.
> > 
> > > regex 1 (/^(.*)\.so\.(.*)$/):
> > > was this meant to capture shared objects with so number and revisons such
> > > as libc.6 or libbz2.so.1.0.4?  If so, should the values after "so" be
> > > restricted to numbers?
> > 
> > It cannot be restricted to numbers only because the version might
> > include other characters.
> I agree now, I did find some examples on my system:
> libtidy.so.5deb1.6.0
> libKPimSMTP.so.5.9.3.abi1


> > This format is the one used commonly for public and stable shared
> > libraries, where the format is <name>.so.<version>, and then there's
> > at least one symlink of the form <name>.so pointing to the former and
> > used by other objects to link to, that gets them the latest <version>.

This part is correct.

> > The key here is that the SONAME might only include part of the
> > SOVERSION, depending on where the <name>.so symlink points to, so you
> > could have:
> > 
> >   libfoo.so.1.2.3
> >   libfoo.so.1
> >   libfoo.so → libfoo.1

This is very confused. The unversioned symlink (or what it resolves to)
does not need to have a direct relation to the SONAME, which is what gets
encoded in the shared library at link time usually via a command-line

So for example (following the one you queried about):

  $ realpath /lib/x86_64-linux-gnu/libc.so.6
  $ objdump -p /lib/x86_64-linux-gnu/libc.so.6 | grep SONAME
  SONAME               libc.so.6

In this case libc uses (for historical reasons) a filename encoding
the project version name (which is distinct to its SOVERSION), but its
SONAME encodes the arch-specific SOVERSION. The actual requirement
here is that the unversioned «<name>.so» used during compile/link-time
refers to the version intended to be linked against, and that the
SONAME can then be found at run-time, directly or via symlinks.

> > Because these are public interfaces they are expected to have
> > dependency information available.

More or less I guess.

> > > regex 2 (/^(.*)-(\d.*)\.so$/)
> > > The terminating so on this regex helps limit its capture but again should
> > > the groups be limited to digits only?

> > This is the format commonly used for private and/or unstable shared
> > libraries, where the whole version is encoded in the SONAME. So you
> > could have:

I think this is also correct in general.

> I was not aware of this distinction.  A quick search did not point me to 
> anything stating that.  Does this mean that libc on my machine is unstable? 
> (libc-2.28.so)

> >   libbar-1.2.3.so
> >   libbar.so → libbar-1.2.3.so (optionally, otherwise you might be
> >                                required to specify the whole SONAME)

An example of this could be:

  $ realpath /usr/lib/x86_64-linux-gnu/libbfd-2.31.1-system.so
  $ objdump -p /usr/lib/x86_64-linux-gnu/libbfd-2.31.1-system.so | grep SONAME
  SONAME               libbfd-2.31.1-system.so

In this case there's no unversioned «<name>.so» symlink, because this
is a private interface, so projects need to know the exact version to
link to.

> > As these are private/unstable interfaces they are to be used at your
> > own risk, so that's why they are just warnings.

This is partial non-sense, sorry. :) The first part holds, but not the

> These do not throw warnings though, they throw errors.  At link time, if the 
> library depended on had an soname that name would be put in the NEEDED field.  
> If the library depended on did not have an soname or any version info then the 
> just the name would be in the NEEDED field.  
> libfoo.so.1 will fail
> libbar-1.2.3.so will fail
> libbaz.so will just be a warning

Yes, sorry. The recognized formats throw errors, the unknown ones
warnings. It is expected that both recognized formats provide
dependency information in the form of shlibs or symbols files.

> So it can't be that dpkg-shlibdeps only wants to throw warnings for private/
> unstable libraries and errors for everything else.  I guess this gets to the 
> heart of my question, why is there a distinction.  If it cannot find a 
> dependency should it just always fail?

Yes, sorry for the confusion. :)

> > I'm not sure now, there's anywhere properly documenting these details.
> > I don't think the debian-policy manual contains rationale for these.
> > I've for now locallt documented the split_soname() function, willcheck
> > whether one of the man pages can also be improved.

(Some heavy typos in there :)


Reply to: