[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [MoM] FAST progress (was Re: [MoM] fast: Add further dependencies to enable chroot / cowbuilder to build)



Hello Andreas,

> Yes.  I'll have a look and may be I'll turn this into a d-shlibs call.
> Thank you have an example.

Sounds good. Although for my attempt I used install command to assign
755 permission to *.so when moved.

I've sometimes found that maintainer consistency is fairly poor.
Although there is a policy in place it seems like sometimes maintainers
go their own ways which is fairly confusing, especially if that
particular thing is not well documented.

I will also push a couple more patches shortly with regards to the
opencl header issue once this builds. That, and the ~rc version mangle.

> May be we finalise this package in the next days.  I'll have a look and
> might finish it if you don't mind.

I'll still be as active as I am for the next couple of weeks, but it's
just an advanced heads up as there is uncertainty with the whole moving
out factor.

Additionally, I am wondering how many packages to maintain before it
gets "too much". Although I guess the hard work is just the initial
packaging attempt. Apart from the whole week I was on holiday, I think
this package just took its time in terms of its size but also
understanding (from scratch) how libraries should be installed, and the
1 hr 30 mins of build times :).

Best Regards,
Shayan Doust

On 20/08/2019 20:21, Andreas Tille wrote:
> Hi Shayan,
> 
> On Tue, Aug 20, 2019 at 07:14:33PM +0100, Shayan Doust wrote:
>>
>> Welcome back :).
> 
> :-)
>  
>> I've probably checked gatb-core at least a dozen times before you sent
>> this to me. I just never checked debian/patch as I didn't think this was
>> some cmake thing :). Talked to upstream and they will take care of
>> soversion within the next release and package update, but for now I just
>> added this as a patch.
> 
> Cool!
>  
>>> ...In this case the dynamic lib is added but you get the idea how to
>>> add a static one from this patch...
>>
>> Thank you. This is done.
>>
>> If you have time, do you mind just checking fast package for me? There
>> are still a few things I need to do with libfast-dev as I haven't done
>> this yet. Where should *.a go? Should this go under
>> /usr/lib/<arch>-linux-gnu?
> 
> Yes.  I'll have a look and may be I'll turn this into a d-shlibs call.
> Thank you have an example.
> 
>> I know this can be set accordingly in
>> debian/rules but I have seen *.a both in /usr/lib and
>> /usr/lib/<arch>-linux-gnu. Same for *.so.
> 
> The latter is the modern (=multiarch) one.  /usr/lib is just a left over
> by not so properly maintaines packages.
> 
>> I also see
>> rc-version-greater-than-expected-version, presumably from how upstream
>> brands the version. Should this be ignored or can this be safely rectified?
> 
> You need to use
> 
>    opts="uversionmangle=s/-rc/~rc/"
> 
> in debian/watch and use ~rc in d/changelog.  The rationale is that '~'
> sorts lower with `dpkg --compare-versions`.  So if upstream is issuing a
> real release it is in correct alphabethic sequence and considered a
> higher version if you drop ~rc (or ~beta or whatever upstream might
> invent to mark a pre-release).
> 
>> Additionally, productivity will start to decrease for only a few weeks
>> to a month as I prepare to move out and into university accommodation.
> 
> May be we finalise this package in the next days.  I'll have a look and
> might finish it if you don't mind.
> 
> Thanks for your great work
> 
>      Andreas.
>  
>> On 19/08/2019 07:39, Andreas Tille wrote:
>>> Hi Shayan,
>>>
>>> On Sun, Aug 18, 2019 at 05:53:01PM +0100, Shayan Doust wrote:
>>>> Hello,
>>>>
>>>> Please disregard the previous email entirely just to save you time from
>>>> writing, as I have finally figured things out.
>>>
>>> I'm slowly recovering from last week need to cope with some backlog.
>>> Thus saving my time is pretty welcome. ;-)
>>>
>>>> fast should contain only executable binaries which in this case are
>>>> openigt fast client and server binaries.
>>>
>>> Yes.
>>>
>>>> libfastSOVERSION should contain the shared / *.so only although grepping
>>>> libFAST.so, there is no soversion. Is this libfast0 then?
>>>
>>> Several upstreams do not know about SOVERSION and once you contact them
>>> you could try to talk about this as well.  If upstream does not set a
>>> soversion we should do this.  In gatb-core you can find a simple example:
>>>
>>>    https://salsa.debian.org/med-team/gatb-core/blob/master/debian/patches/set_soversion.patch
>>>
>>> Yes, 0 is a good choice here and we need to bump the SOVERSION once
>>> upstream might change the ABI (which frequently happens without any
>>> notice - you see we are at version 2 in gatb-core meanwhile).
>>>
>>>> libfast-dev contains only the header files and the static library (*.a).
>>>> I realised this can be done with the ar tool. Is it sensible to traverse
>>>> the build directory and add all object files to archive to create a
>>>> libfast.a?
>>>
>>> There is no need for manual intervention with ar.  CMake can do this
>>> easily.  Once we have used gatb-core as an example we might stick to
>>> this one:
>>>
>>>    https://salsa.debian.org/med-team/gatb-core/blob/master/debian/patches/dynamic_lib.patch
>>>
>>> In this case the dynamic lib is added but you get the idea how to
>>> add a static one from this patch.
>>>  
>>>> I will now just slightly modify the existing debian/ files to
>>>> accommodate these changes.
>>>>
>>>> Additionally, I will stick to modifying fast so I will not need the
>>>> additional CL header files, so the dependency is now just opencl. I will
>>>> talk to upstream about this.
>>>
>>> If this approach will work out that sounds like a good alternative
>>> which was missing in my list.
>>>
>>>> Many thanks & hopefully this isn't an inconvenience,
>>>
>>> Definitely not.  Its a pleasure to see you growing into way more
>>> complex packages.
>>>
>>> Kind regards
>>>
>>>        Andreas.
>>>
>>>> Shayan Doust
>>>>
>>>>
>>>> -------- Forwarded Message --------
>>>> Subject: Re: [MoM] Re: fast: Add further dependencies to enable chroot /
>>>> cowbuilder to build
>>>> Resent-Date: Sat, 17 Aug 2019 13:16:24 +0000 (UTC)
>>>> Resent-From: debian-med@lists.debian.org
>>>> Date: Sat, 17 Aug 2019 14:16:05 +0100
>>>> From: Shayan Doust <hello@shayandoust.me>
>>>> To: debian-med@lists.debian.org
>>>>
>>>> Hello Andreas,
>>>>
>>>> Many thanks for taking the time to reply.
>>>>
>>>> Just a few side comments for whenever you next have the free time to
>>>> reply :).
>>>>
>>>>> I had no time to check the packaging.  In any case I think if this is
>>>>> a library package we usually ship libfast-dev (with header files and
>>>>> static lib (*.a file(s) )), a dynamic library package libfastSOVERSION
>>>>> and the executable in a fast package.
>>>>
>>>> I retained the original name as the package name "fast" and not
>>>> "libfast". Maybe I should change this. Initially I was unsure as fast
>>>> stands to framework and was unsure as to this relationship with a library.
>>>>
>>>> CMake only generates FAST as a shared object / *.so. I assume this is
>>>> just what upstream wanted. Referring to deb policy 8.3, I don't see this
>>>> as mandatory. If it is, maybe I'd need to use some tool or modify cmake
>>>> to output *.a. Fast did however generate example binaries to which I
>>>> split this off into fast-examples to install under /usr/lib/fast.
>>>>
>>>> There is no soversion when doing an objdump and grepping the SONAME from
>>>> the generated *.so file. Does this therefore mean a soversion of simply
>>>> 0 or does this now reflect upstream version. I think it's simply a
>>>> mistake to simply move libFAST.so into /usr/lib as this doesn't have a
>>>> soversion and has to be a symlink instead.
>>>>
>>>> I may have picked a fairly time consuming and confusing package until I
>>>> wrap my head around how a library / "framework" should be packaged.
>>>>
>>>> Best Regards,
>>>> Shayan Doust
>>>>
>>>> On 17/08/2019 10:17, Andreas Tille wrote:
>>>>> Hi Shayan,
>>>>>
>>>>> I've found some spare minutes for a more explicit answer.  This week I
>>>>> was pretty busy with real life (nursing my two grandsons is pretty much
>>>>> a full time job :-P).
>>>>>
>>>>> On Wed, Aug 14, 2019 at 04:49:00PM +0100, Shayan Doust wrote:
>>>>>> Hello,
>>>>>>
>>>>>> So I contacted upstream regarding the failed test binary generation and
>>>>>> they've acknowledged and fixed it. A query regarding test data needed
>>>>>> for autopkgtest. As you said to avoid git or any downloading tools
>>>>>> (curl, wget, ...) as a dependency,
>>>>>
>>>>> Its not a matter of trying to avoid some of these tools.  Per policy a
>>>>> package needs to build on a machine that's disconnected from the
>>>>> internet.  So it will just not work technically to download anything.
>>>>> Its the same with autopkgtests.  So we need to provide everything we
>>>>> need either as Debian package or inside the source tree.
>>>>>
>>>>>> how can I add the test datas to
>>>>>> autopkgtest. The test data is zipped and is just over 2 GB large, so I
>>>>>> didn't think patching this in would be sensible. The test data are to be
>>>>>> downloaded from https://folk.idi.ntnu.no/smistad/FAST_Test_Data.zip
>>>>>
>>>>> I think 2GB data are to much to just move it to the debian/ dir.  So the
>>>>> best idea would be to provide the test data in a separate package for
>>>>> instance fast-data (or fast-test-data / fast-examples).  
>>>>>
>>>>>> Additionally, I've got another query regarding opencl. Upstream have
>>>>>> their own modified version of the CL headers. Using diff, the only
>>>>>> change they have done is add two *.hpp files into the CL header
>>>>>> directory in /usr/include. Is it sensible to ever have opencl as a
>>>>>> prerequisite / package dependency and then move over the two missing
>>>>>> files into the CL directory when the user installs the fast package or
>>>>>> should this sort of modification to external packages be avoided at all
>>>>>> costs. I assume the other way would just be to have the fast opencl
>>>>>> headers inside /usr/include/FAST and then patch all the fast headers to
>>>>>> use the fast opencl headers in the new directory. CMake also generated
>>>>>> some opencl *.cl files in a directory called "kernel" so I am not sure
>>>>>> as to what I should do with this directory and its significance to FAST.
>>>>>
>>>>> I agree that having a full copy of OpenCL just to ship two extra files
>>>>> makes no sense.  I see several options to consider:
>>>>>
>>>>>   1. May be it makes sense to forward these two files to OpenCL upstream.
>>>>>      In any case it might make sense to talk to fast upstream / opencl
>>>>>      Debian maintainers.
>>>>>
>>>>> You proposed yourself which does not involve others and thus is a faster
>>>>> solution for the moment
>>>>>
>>>>>   2. Move these missing files inside a libfast-dev package and feed
>>>>>      it into the opencl headers.  I admit while this would work I have
>>>>>      a bad gut feeling about this.
>>>>>
>>>>>   3. Patch the files and ship the two additional files somewhere in
>>>>>      /usr/include/fast
>>>>>
>>>>> I admit I prefer option 3. as a temporaty means until may be 1. can
>>>>> be implemented.
>>>>>  
>>>>>> Looks like everything else is going fine. With this being the first
>>>>>> library I've packaged I do expect some mistakes but luckily they won't
>>>>>> be replicated within the next library I package :).
>>>>>
>>>>> I had no time to check the packaging.  In any case I think if this is
>>>>> a library package we usually ship libfast-dev (with header files and
>>>>> static lib (*.a file(s) )), a dynamic library package libfastSOVERSION
>>>>> and the executable in a fast package.
>>>>>
>>>>> I have no time to verify whether these hints might make sense in this
>>>>> actual case.
>>>>>
>>>>> Kind regards
>>>>>
>>>>>       Andreas.
>>>>>
>>>>>> On 11/08/2019 21:49, Shayan Doust wrote:
>>>>>>> Hi Andreas,
>>>>>>>
>>>>>>>> I'm occupied by real life until next weekend - so my response time
>>>>>>>> is way longer than usual.
>>>>>>>
>>>>>>> Not a worry at all & thanks for the needed information!
>>>>>>>
>>>>>>>> May be you can ask on debian-mentors@lists.debian.org meanwhile.
>>>>>>>
>>>>>>> As this is 3.0.0rc1 I will probably try out 3.0.0rc3 and then ask just
>>>>>>> in case this was some upstream issue.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Shayan Doust
>>>>>>>
>>>>>>> On 11/08/2019 21:44, Andreas Tille wrote:
>>>>>>>> Hi Shayan,
>>>>>>>>
>>>>>>>> I'm occupied by real life until next weekend - so my response time
>>>>>>>> is way longer than usual.
>>>>>>>>
>>>>>>>> On Sun, Aug 11, 2019 at 01:25:22AM +0100, Shayan Doust wrote:
>>>>>>>>> Hello Andreas,
>>>>>>>>>
>>>>>>>>> A few things changed since the previous email. I found a way of getting
>>>>>>>>> the dependencies via another chroot environment and now the package
>>>>>>>>> builds in cowbuilder with no troubles. My work routine usually goes
>>>>>>>>> along the lines of getting stuck on something for a couple of hours,
>>>>>>>>> emailing here on the mailing list then 30 mins later somehow managing to
>>>>>>>>> fix whatever issue I was stuck on :).
>>>>>>>>
>>>>>>>> That's not very different to what happened quite frequently to me. ;-)
>>>>>>>>
>>>>>>>>> I am having some issues with moving libFAST.so. I am not sure if I
>>>>>>>>> should simply use mv or use d-shlibmove. d-shlibmove just throws an
>>>>>>>>> error with regards to dependencies not existing so if I am meant to use
>>>>>>>>> d-shlibmove, please have a look at this in fast.
>>>>>>>>
>>>>>>>> I personally like to use d-shlibmove since it prevents you from making
>>>>>>>> several mistakes in library packaging.  However, since I habe no time
>>>>>>>> to provide technical help this week its fine if you find any solution.
>>>>>>>> Usually d-shlibmove turns out a bit tricky.  If something is missing
>>>>>>>> you can try '--override' as for instance in the package libdisorder.
>>>>>>>>
>>>>>>>>> Lintian is reporting package-name-doesnt-match-sonames. I believe this
>>>>>>>>> is where I have to rename "fast" to "libFAST" for the package name. I am
>>>>>>>>> also getting shlib-without-versioned-soname and I am unsure as to how
>>>>>>>>> this is rectified.
>>>>>>>>
>>>>>>>> I usually ignore these lintian issues when its not an actual library
>>>>>>>> package.
>>>>>>>>  
>>>>>>>>> Compilation of the test binaries fail and I am even unable to build
>>>>>>>>> these in an isolated system just using cmake so I assume this is some
>>>>>>>>> sort of an upstream bug or even an incomplete wiki page with some
>>>>>>>>> dependency not documented. I'll figure out something for this as usual.
>>>>>>>>> Luckily all other informational lintian outputs can simply be fixed by
>>>>>>>>> removing the unneeded directories like fonts. I can't think of anything
>>>>>>>>> else to write at this time of night.
>>>>>>>>
>>>>>>>> May be you can ask on debian-mentors@lists.debian.org meanwhile.
>>>>>>>>  
>>>>>>>>> Thanks for your time & best regards,
>>>>>>>>
>>>>>>>> Good luck
>>>>>>>>
>>>>>>>>      Andreas.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
> 
> 
> 
> 

Attachment: signature.asc
Description: OpenPGP digital signature


Reply to: