[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [MoM] FAST progress (was Re: [MoM] fast: Add further dependencies to enable chroot / cowbuilder to build)



Hello Andreas,

For multiarch, I decided to move the shared object and static lib into
directory defined by DEB_HOST_MULTIARCH.

As a result, during the dpkg_shlibdeps invoke when dh_shlibdeps is
reached in the package build process, I see a lot of errors. A snippet
is denoted as below:

dpkg-shlibdeps: error: cannot continue due to the errors listed above
Note: libraries are not searched in other binary packages that do not
have any shlibs or symbols file.
To help dpkg-shlibdeps find private libraries, you might need to use -l.
dh_shlibdeps: dpkg-shlibdeps -Tdebian/fast.substvars
debian/fast/usr/lib/fast/OpenIGTLinkClient
debian/fast/usr/lib/fast/OpenIGTLinkServer returned exit code 2
dpkg-shlibdeps: error: cannot find library libFAST.so.0 needed by
debian/libfast-examples/usr/lib/fast/streamImagesFromDisk (ELF format:
'elf64-x86-64' abi: '0201003e00000000'; RPATH: '/usr/lib/fast/../lib')

Is the RPATH to blame? I'm not too sure what I should do and I would
rather not trial and error with how long each build takes. The latest
commit reflects the state of my current workspace. Maybe I should have
looked into using include(GNUInstallDirs).

Best Regards,
Shayan Doust

On 20/08/2019 20:42, Shayan Doust wrote:
> Hello Andreas,
> 
>> Yes.  I'll have a look and may be I'll turn this into a d-shlibs call.
>> Thank you have an example.
> 
> Sounds good. Although for my attempt I used install command to assign
> 755 permission to *.so when moved.
> 
> I've sometimes found that maintainer consistency is fairly poor.
> Although there is a policy in place it seems like sometimes maintainers
> go their own ways which is fairly confusing, especially if that
> particular thing is not well documented.
> 
> I will also push a couple more patches shortly with regards to the
> opencl header issue once this builds. That, and the ~rc version mangle.
> 
>> May be we finalise this package in the next days.  I'll have a look and
>> might finish it if you don't mind.
> 
> I'll still be as active as I am for the next couple of weeks, but it's
> just an advanced heads up as there is uncertainty with the whole moving
> out factor.
> 
> Additionally, I am wondering how many packages to maintain before it
> gets "too much". Although I guess the hard work is just the initial
> packaging attempt. Apart from the whole week I was on holiday, I think
> this package just took its time in terms of its size but also
> understanding (from scratch) how libraries should be installed, and the
> 1 hr 30 mins of build times :).
> 
> Best Regards,
> Shayan Doust
> 
> On 20/08/2019 20:21, Andreas Tille wrote:
>> Hi Shayan,
>>
>> On Tue, Aug 20, 2019 at 07:14:33PM +0100, Shayan Doust wrote:
>>>
>>> Welcome back :).
>>
>> :-)
>>  
>>> I've probably checked gatb-core at least a dozen times before you sent
>>> this to me. I just never checked debian/patch as I didn't think this was
>>> some cmake thing :). Talked to upstream and they will take care of
>>> soversion within the next release and package update, but for now I just
>>> added this as a patch.
>>
>> Cool!
>>  
>>>> ...In this case the dynamic lib is added but you get the idea how to
>>>> add a static one from this patch...
>>>
>>> Thank you. This is done.
>>>
>>> If you have time, do you mind just checking fast package for me? There
>>> are still a few things I need to do with libfast-dev as I haven't done
>>> this yet. Where should *.a go? Should this go under
>>> /usr/lib/<arch>-linux-gnu?
>>
>> Yes.  I'll have a look and may be I'll turn this into a d-shlibs call.
>> Thank you have an example.
>>
>>> I know this can be set accordingly in
>>> debian/rules but I have seen *.a both in /usr/lib and
>>> /usr/lib/<arch>-linux-gnu. Same for *.so.
>>
>> The latter is the modern (=multiarch) one.  /usr/lib is just a left over
>> by not so properly maintaines packages.
>>
>>> I also see
>>> rc-version-greater-than-expected-version, presumably from how upstream
>>> brands the version. Should this be ignored or can this be safely rectified?
>>
>> You need to use
>>
>>    opts="uversionmangle=s/-rc/~rc/"
>>
>> in debian/watch and use ~rc in d/changelog.  The rationale is that '~'
>> sorts lower with `dpkg --compare-versions`.  So if upstream is issuing a
>> real release it is in correct alphabethic sequence and considered a
>> higher version if you drop ~rc (or ~beta or whatever upstream might
>> invent to mark a pre-release).
>>
>>> Additionally, productivity will start to decrease for only a few weeks
>>> to a month as I prepare to move out and into university accommodation.
>>
>> May be we finalise this package in the next days.  I'll have a look and
>> might finish it if you don't mind.
>>
>> Thanks for your great work
>>
>>      Andreas.
>>  
>>> On 19/08/2019 07:39, Andreas Tille wrote:
>>>> Hi Shayan,
>>>>
>>>> On Sun, Aug 18, 2019 at 05:53:01PM +0100, Shayan Doust wrote:
>>>>> Hello,
>>>>>
>>>>> Please disregard the previous email entirely just to save you time from
>>>>> writing, as I have finally figured things out.
>>>>
>>>> I'm slowly recovering from last week need to cope with some backlog.
>>>> Thus saving my time is pretty welcome. ;-)
>>>>
>>>>> fast should contain only executable binaries which in this case are
>>>>> openigt fast client and server binaries.
>>>>
>>>> Yes.
>>>>
>>>>> libfastSOVERSION should contain the shared / *.so only although grepping
>>>>> libFAST.so, there is no soversion. Is this libfast0 then?
>>>>
>>>> Several upstreams do not know about SOVERSION and once you contact them
>>>> you could try to talk about this as well.  If upstream does not set a
>>>> soversion we should do this.  In gatb-core you can find a simple example:
>>>>
>>>>    https://salsa.debian.org/med-team/gatb-core/blob/master/debian/patches/set_soversion.patch
>>>>
>>>> Yes, 0 is a good choice here and we need to bump the SOVERSION once
>>>> upstream might change the ABI (which frequently happens without any
>>>> notice - you see we are at version 2 in gatb-core meanwhile).
>>>>
>>>>> libfast-dev contains only the header files and the static library (*.a).
>>>>> I realised this can be done with the ar tool. Is it sensible to traverse
>>>>> the build directory and add all object files to archive to create a
>>>>> libfast.a?
>>>>
>>>> There is no need for manual intervention with ar.  CMake can do this
>>>> easily.  Once we have used gatb-core as an example we might stick to
>>>> this one:
>>>>
>>>>    https://salsa.debian.org/med-team/gatb-core/blob/master/debian/patches/dynamic_lib.patch
>>>>
>>>> In this case the dynamic lib is added but you get the idea how to
>>>> add a static one from this patch.
>>>>  
>>>>> I will now just slightly modify the existing debian/ files to
>>>>> accommodate these changes.
>>>>>
>>>>> Additionally, I will stick to modifying fast so I will not need the
>>>>> additional CL header files, so the dependency is now just opencl. I will
>>>>> talk to upstream about this.
>>>>
>>>> If this approach will work out that sounds like a good alternative
>>>> which was missing in my list.
>>>>
>>>>> Many thanks & hopefully this isn't an inconvenience,
>>>>
>>>> Definitely not.  Its a pleasure to see you growing into way more
>>>> complex packages.
>>>>
>>>> Kind regards
>>>>
>>>>        Andreas.
>>>>
>>>>> Shayan Doust
>>>>>
>>>>>
>>>>> -------- Forwarded Message --------
>>>>> Subject: Re: [MoM] Re: fast: Add further dependencies to enable chroot /
>>>>> cowbuilder to build
>>>>> Resent-Date: Sat, 17 Aug 2019 13:16:24 +0000 (UTC)
>>>>> Resent-From: debian-med@lists.debian.org
>>>>> Date: Sat, 17 Aug 2019 14:16:05 +0100
>>>>> From: Shayan Doust <hello@shayandoust.me>
>>>>> To: debian-med@lists.debian.org
>>>>>
>>>>> Hello Andreas,
>>>>>
>>>>> Many thanks for taking the time to reply.
>>>>>
>>>>> Just a few side comments for whenever you next have the free time to
>>>>> reply :).
>>>>>
>>>>>> I had no time to check the packaging.  In any case I think if this is
>>>>>> a library package we usually ship libfast-dev (with header files and
>>>>>> static lib (*.a file(s) )), a dynamic library package libfastSOVERSION
>>>>>> and the executable in a fast package.
>>>>>
>>>>> I retained the original name as the package name "fast" and not
>>>>> "libfast". Maybe I should change this. Initially I was unsure as fast
>>>>> stands to framework and was unsure as to this relationship with a library.
>>>>>
>>>>> CMake only generates FAST as a shared object / *.so. I assume this is
>>>>> just what upstream wanted. Referring to deb policy 8.3, I don't see this
>>>>> as mandatory. If it is, maybe I'd need to use some tool or modify cmake
>>>>> to output *.a. Fast did however generate example binaries to which I
>>>>> split this off into fast-examples to install under /usr/lib/fast.
>>>>>
>>>>> There is no soversion when doing an objdump and grepping the SONAME from
>>>>> the generated *.so file. Does this therefore mean a soversion of simply
>>>>> 0 or does this now reflect upstream version. I think it's simply a
>>>>> mistake to simply move libFAST.so into /usr/lib as this doesn't have a
>>>>> soversion and has to be a symlink instead.
>>>>>
>>>>> I may have picked a fairly time consuming and confusing package until I
>>>>> wrap my head around how a library / "framework" should be packaged.
>>>>>
>>>>> Best Regards,
>>>>> Shayan Doust
>>>>>
>>>>> On 17/08/2019 10:17, Andreas Tille wrote:
>>>>>> Hi Shayan,
>>>>>>
>>>>>> I've found some spare minutes for a more explicit answer.  This week I
>>>>>> was pretty busy with real life (nursing my two grandsons is pretty much
>>>>>> a full time job :-P).
>>>>>>
>>>>>> On Wed, Aug 14, 2019 at 04:49:00PM +0100, Shayan Doust wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> So I contacted upstream regarding the failed test binary generation and
>>>>>>> they've acknowledged and fixed it. A query regarding test data needed
>>>>>>> for autopkgtest. As you said to avoid git or any downloading tools
>>>>>>> (curl, wget, ...) as a dependency,
>>>>>>
>>>>>> Its not a matter of trying to avoid some of these tools.  Per policy a
>>>>>> package needs to build on a machine that's disconnected from the
>>>>>> internet.  So it will just not work technically to download anything.
>>>>>> Its the same with autopkgtests.  So we need to provide everything we
>>>>>> need either as Debian package or inside the source tree.
>>>>>>
>>>>>>> how can I add the test datas to
>>>>>>> autopkgtest. The test data is zipped and is just over 2 GB large, so I
>>>>>>> didn't think patching this in would be sensible. The test data are to be
>>>>>>> downloaded from https://folk.idi.ntnu.no/smistad/FAST_Test_Data.zip
>>>>>>
>>>>>> I think 2GB data are to much to just move it to the debian/ dir.  So the
>>>>>> best idea would be to provide the test data in a separate package for
>>>>>> instance fast-data (or fast-test-data / fast-examples).  
>>>>>>
>>>>>>> Additionally, I've got another query regarding opencl. Upstream have
>>>>>>> their own modified version of the CL headers. Using diff, the only
>>>>>>> change they have done is add two *.hpp files into the CL header
>>>>>>> directory in /usr/include. Is it sensible to ever have opencl as a
>>>>>>> prerequisite / package dependency and then move over the two missing
>>>>>>> files into the CL directory when the user installs the fast package or
>>>>>>> should this sort of modification to external packages be avoided at all
>>>>>>> costs. I assume the other way would just be to have the fast opencl
>>>>>>> headers inside /usr/include/FAST and then patch all the fast headers to
>>>>>>> use the fast opencl headers in the new directory. CMake also generated
>>>>>>> some opencl *.cl files in a directory called "kernel" so I am not sure
>>>>>>> as to what I should do with this directory and its significance to FAST.
>>>>>>
>>>>>> I agree that having a full copy of OpenCL just to ship two extra files
>>>>>> makes no sense.  I see several options to consider:
>>>>>>
>>>>>>   1. May be it makes sense to forward these two files to OpenCL upstream.
>>>>>>      In any case it might make sense to talk to fast upstream / opencl
>>>>>>      Debian maintainers.
>>>>>>
>>>>>> You proposed yourself which does not involve others and thus is a faster
>>>>>> solution for the moment
>>>>>>
>>>>>>   2. Move these missing files inside a libfast-dev package and feed
>>>>>>      it into the opencl headers.  I admit while this would work I have
>>>>>>      a bad gut feeling about this.
>>>>>>
>>>>>>   3. Patch the files and ship the two additional files somewhere in
>>>>>>      /usr/include/fast
>>>>>>
>>>>>> I admit I prefer option 3. as a temporaty means until may be 1. can
>>>>>> be implemented.
>>>>>>  
>>>>>>> Looks like everything else is going fine. With this being the first
>>>>>>> library I've packaged I do expect some mistakes but luckily they won't
>>>>>>> be replicated within the next library I package :).
>>>>>>
>>>>>> I had no time to check the packaging.  In any case I think if this is
>>>>>> a library package we usually ship libfast-dev (with header files and
>>>>>> static lib (*.a file(s) )), a dynamic library package libfastSOVERSION
>>>>>> and the executable in a fast package.
>>>>>>
>>>>>> I have no time to verify whether these hints might make sense in this
>>>>>> actual case.
>>>>>>
>>>>>> Kind regards
>>>>>>
>>>>>>       Andreas.
>>>>>>
>>>>>>> On 11/08/2019 21:49, Shayan Doust wrote:
>>>>>>>> Hi Andreas,
>>>>>>>>
>>>>>>>>> I'm occupied by real life until next weekend - so my response time
>>>>>>>>> is way longer than usual.
>>>>>>>>
>>>>>>>> Not a worry at all & thanks for the needed information!
>>>>>>>>
>>>>>>>>> May be you can ask on debian-mentors@lists.debian.org meanwhile.
>>>>>>>>
>>>>>>>> As this is 3.0.0rc1 I will probably try out 3.0.0rc3 and then ask just
>>>>>>>> in case this was some upstream issue.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Shayan Doust
>>>>>>>>
>>>>>>>> On 11/08/2019 21:44, Andreas Tille wrote:
>>>>>>>>> Hi Shayan,
>>>>>>>>>
>>>>>>>>> I'm occupied by real life until next weekend - so my response time
>>>>>>>>> is way longer than usual.
>>>>>>>>>
>>>>>>>>> On Sun, Aug 11, 2019 at 01:25:22AM +0100, Shayan Doust wrote:
>>>>>>>>>> Hello Andreas,
>>>>>>>>>>
>>>>>>>>>> A few things changed since the previous email. I found a way of getting
>>>>>>>>>> the dependencies via another chroot environment and now the package
>>>>>>>>>> builds in cowbuilder with no troubles. My work routine usually goes
>>>>>>>>>> along the lines of getting stuck on something for a couple of hours,
>>>>>>>>>> emailing here on the mailing list then 30 mins later somehow managing to
>>>>>>>>>> fix whatever issue I was stuck on :).
>>>>>>>>>
>>>>>>>>> That's not very different to what happened quite frequently to me. ;-)
>>>>>>>>>
>>>>>>>>>> I am having some issues with moving libFAST.so. I am not sure if I
>>>>>>>>>> should simply use mv or use d-shlibmove. d-shlibmove just throws an
>>>>>>>>>> error with regards to dependencies not existing so if I am meant to use
>>>>>>>>>> d-shlibmove, please have a look at this in fast.
>>>>>>>>>
>>>>>>>>> I personally like to use d-shlibmove since it prevents you from making
>>>>>>>>> several mistakes in library packaging.  However, since I habe no time
>>>>>>>>> to provide technical help this week its fine if you find any solution.
>>>>>>>>> Usually d-shlibmove turns out a bit tricky.  If something is missing
>>>>>>>>> you can try '--override' as for instance in the package libdisorder.
>>>>>>>>>
>>>>>>>>>> Lintian is reporting package-name-doesnt-match-sonames. I believe this
>>>>>>>>>> is where I have to rename "fast" to "libFAST" for the package name. I am
>>>>>>>>>> also getting shlib-without-versioned-soname and I am unsure as to how
>>>>>>>>>> this is rectified.
>>>>>>>>>
>>>>>>>>> I usually ignore these lintian issues when its not an actual library
>>>>>>>>> package.
>>>>>>>>>  
>>>>>>>>>> Compilation of the test binaries fail and I am even unable to build
>>>>>>>>>> these in an isolated system just using cmake so I assume this is some
>>>>>>>>>> sort of an upstream bug or even an incomplete wiki page with some
>>>>>>>>>> dependency not documented. I'll figure out something for this as usual.
>>>>>>>>>> Luckily all other informational lintian outputs can simply be fixed by
>>>>>>>>>> removing the unneeded directories like fonts. I can't think of anything
>>>>>>>>>> else to write at this time of night.
>>>>>>>>>
>>>>>>>>> May be you can ask on debian-mentors@lists.debian.org meanwhile.
>>>>>>>>>  
>>>>>>>>>> Thanks for your time & best regards,
>>>>>>>>>
>>>>>>>>> Good luck
>>>>>>>>>
>>>>>>>>>      Andreas.
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>>
> 

Attachment: signature.asc
Description: OpenPGP digital signature


Reply to: