[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [MoM] FAST progress (was Re: [MoM] fast: Add further dependencies to enable chroot / cowbuilder to build)



Hello Andreas,

>   Unable to open directory /usr/../doc//fonts

That's very annoying. The docs file is expected to be installed in /usr
and contains "FAST" image branding and some other assets that are used
in the program. I will see if FAST allows for dynamic pathing and where
it actually wants me to install the doc folder. The reason I did not
install the doc folder under /usr/share and excluded it for now is
because it serves no purpose to the end user. So now I will look into it
and if not, introduce another patch to cater for the doc directory to be
under a different location.

SIGSEVG probably because it expects these "doc" assets. Luckily there is
no early assertion of CL which means the CL patch probably worked. I
will try offloading this onto my build server, and manually testing the
test suite on another spare laptop and hopefully opencl is supported.

> A general remark:  The package libfast-examples contains compiled
> executables in /usr/lib/fast.  Users usually expect examples in
>
>     /usr/share/doc/PKGNAME/examples
>
> (that's where dh_installexamples puts files).  Usually these are not
> binaries.  It depends a bit what we want to teach users by these
> examples.  If it is how to develop with libfast-dev than it would be
> better to provide the source code files including a Makefile to compile
> these examples (and may be change something and try what's happening).
>
> If something else should be demonstrated and these executables might
> have some extra value I'd consider it necessary to add some information
> under /usr/share/doc/libfast-examples/examples or similar to inform
> users where they can find those files and what to do.

Ahh I see. I am still fairly uninformed as to what these examples
represent (until I rectify this opencl issue). I thought these binaries
would be nice because they represent the examples on the wiki[1].

> If it is how to develop with libfast-dev than it would be
> better to provide the source code files including a Makefile to
> compile these examples (and may be change something and try what's
> happening).

That could work, although that would mean writing the examples and
writing my own makefile which could work as this is not readily
available, however I think a simpler option would be the second point
you suggested. Maybe adding a documentation / some file under
/usr/share/doc/libfast-examples/examples to both explain the purpose,
refer to the wiki and the new example location. I will play with these
ideas once I figure out this opencl.

Kind regards & thanks for your help,
Shayan Doust

[1]: https://github.com/smistad/FAST/wiki/Examples

On 30/08/2019 17:05, Andreas Tille wrote:
> Hi Shayan,
> 
> please git pull for two commits (library name to make sure fast-examples
> can be installed and Standards-Version).
> 
> On Thu, Aug 29, 2019 at 01:22:51AM +0100, Shayan Doust wrote:
>> Hello Andreas,
>>
>>> In libhmsbeagle I had to deal with this in build time tests.  May be
>>> you check the Build-Depends for some additional inspiration.  No idea
>>> whether this might help.
>>
>> Many thanks for your reply. This has made some slight difference. A few
>> things to note.
>>
>> Running clinfo returns my CPU as a platform, however "what():
>> clGetPlatformIDs" is still thrown. Running the test binary, I see some
>> more information. I am not sure the significance of these as opposed to
>> the separate example binaries, however here is a short snippet:
>>
>> ...
>>
>> -------------------------------------------------------------------------------
>> /home/shayandoust/Desktop/FAST/fast/source/FAST/Tests/Benchmarks.cpp:106
>> ...............................................................................
>>
>> /home/shayandoust/Desktop/FAST/fast/source/FAST/Tests/Benchmarks.cpp:106: FAILED:
>> due to unexpected exception with message:
>>   There was an error while getting OpenCL devices: clGetDeviceIDs
>>
>> INFO: QApp already exists..
>> INFO: Device manager initialize..
>> INFO: Initial GLX context 0x5574dc453ae0
>> INFO: Found 1 OpenCL platforms.
>> INFO: 1 platforms selected for inspection.
>> INFO: Platform 0: Mesa
>> INFO: This platform has
>> -------------------------------------------------------------------------------
>> Pipeline C
>> -------------------------------------------------------------------------------
>> /home/shayandoust/Desktop/FAST/fast/source/FAST/Tests/Benchmarks.cpp:167
>> ...............................................................................
>>
>> /home/shayandoust/Desktop/FAST/fast/source/FAST/Tests/Benchmarks.cpp:167: FAILED:
>> due to unexpected exception with message:
>>   There was an error while getting OpenCL devices: clGetDeviceIDs
>>
>> ===============================================================================
>> test cases: 221 |  32 passed | 189 failed
>> assertions: 385 | 196 passed | 189 failed
> 
> I get:
> 
> $ sh run-unit-test 
> Unpacking test data
> Invoking testFAST
> INFO: Creating new QApp
> WARNING: Unable to open the configuration file /usr/fast_configuration.txt. Using defaults instead.
> 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> testFAST is a Catch v1.5.9 host application.
> Run with -? for options
> 
> -------------------------------------------------------------------------------
> Airway segmentation
> -------------------------------------------------------------------------------
> /build/fast-3.0.0~rc3+ds/source/FAST/Algorithms/AirwaySegmentation/airwaySegmentationTests.cpp:49
> ...............................................................................
> 
> /build/fast-3.0.0~rc3+ds/source/FAST/Algorithms/AirwaySegmentation/airwaySegmentationTests.cpp:49: FAILED:
> due to unexpected exception with message:
>   Unable to open directory /usr/../doc//fonts
> 
> INFO: QApp already exists..
> INFO: Creating new GL context for computation thread
> WARNING: Your system uses comma as decimal point.
> WARNING: This will now be changed to dot to avoid any comma related bugs.
> INFO: Device manager initialize..
> INFO: Initial GLX context 0x563a2ac8d8c0
> INFO: Found 1 OpenCL platforms.
> INFO: 1 platforms selected for inspection.
> INFO: Platform 0: Intel
> INFO: This platform has 0 available devices in total
> INFO: Looking for GPU devices only.
> INFO: 1 selected.
> INFO: Inspecting device 0 with the name Intel(R) HD Graphics Haswell Ultrabook GT2 Mobile
> INFO: Initial GLX context 0x563a2ac8d8c0
> -------------------------------------------------------------------------------
> Block matching 2D
> -------------------------------------------------------------------------------
> /build/fast-3.0.0~rc3+ds/source/FAST/Algorithms/BlockMatching/Tests.cpp:10
> ...............................................................................
> 
> /build/fast-3.0.0~rc3+ds/source/FAST/Algorithms/BlockMatching/Tests.cpp:10: FAILED:
> due to a fatal error condition:
>   SIGSEGV - Segmentation violation signal
> 
> ===============================================================================
> test cases: 2 | 2 failed
> assertions: 2 | 2 failed
> 
>> spent a few hours reading the source and attempting to search up the
>> error message, although I'm trying not to put any additional load on
>> you. If you do lack time, I could always ask in mentors.
> 
> The missing font is a bit suspicious but I'm absolutely uninformed
> about all this graphics stuff to be more helpful here.
> 
> 
> A general remark:  The package libfast-examples contains compiled
> executables in /usr/lib/fast.  Users usually expect examples in
> 
>     /usr/share/doc/PKGNAME/examples
> 
> (that's where dh_installexamples puts files).  Usually these are not
> binaries.  It depends a bit what we want to teach users by these
> examples.  If it is how to develop with libfast-dev than it would be
> better to provide the source code files including a Makefile to compile
> these examples (and may be change something and try what's happening).
> 
> If something else should be demonstrated and these executables might
> have some extra value I'd consider it necessary to add some information
> under /usr/share/doc/libfast-examples/examples or similar to inform
> users where they can find those files and what to do.
> 
> Kind regards
> 
>        Andreas.
>  
>> Many thanks,
>> Shayan Doust
>>
>> On 28/08/2019 20:04, Andreas Tille wrote:
>>> On Wed, Aug 28, 2019 at 06:18:59PM +0100, Shayan Doust wrote:
>>>> Just as I anticipated, it looks like it will not work as I use
>>>> virtualbox and debian is essentially virtualised. So not only can I not
>>>> passthrough the host GPU, I am unable to use any drivers which means
>>>> that the error I get even when running any of the fast-examples binaries
>>>> (apart from OpenIGTLinkServer) is "what(): clGetPlatformIDs", which is
>>>> verified by running "clinfo" which returns 0 as the number of platforms
>>>> - it seems like I have hit a dead end unless there is some work around
>>>> which I have my doubts in.
>>>
>>> In libhmsbeagle I had to deal with this in build time tests.  May be
>>> you check the Build-Depends for some additional inspiration.  No idea
>>> whether this might help.
>>>
>>> Kind regards
>>>
>>>      Andreas (offline for the next 20 hours)
>>>  
>>>> Kind regards,
>>>> Shayan Doust
>>>>
>>>> On 28/08/2019 16:33, Shayan Doust wrote:
>>>>> Hello Andreas,
>>>>>
>>>>>> Ahhh, that really saves time.  I had an extremely busy weekend -
>>>>>> otherwise I would have answered before.
>>>>>
>>>>> No problem. At least I prevented you from answering an old question :).
>>>>>
>>>>>> The symbols file changes are really strange.  I was running again in a
>>>>>> diff.  I have not commited my changes yet but for the moment I have
>>>>>> deactivated the symbols file since it seems we are playing diff - patch
>>>>>> ping-pong for no good reason.
>>>>>
>>>>> I haven't seen this happen with other libraries so it is strange.
>>>>> Perhaps our environments are different in some way? It still shouldn't
>>>>> matter though as the library itself is one version and not changing.
>>>>>
>>>>>> I admit I have never seen this.  D-shlibs enforces
>>>>>>     (= ${binary:Version})
>>>>>> and this should be there.  Your.  May be your change was needed once
>>>>>> the package was arch: all ?
>>>>>
>>>>> You're correct. I changed the depends before I realised the arch was
>>>>> incorrect, and as usual I forgot to backtrack and undo my change to
>>>>> depends. I've just pushed this change.
>>>>>
>>>>>> Testing with opengl is always a nuisance.  Even with emulation this
>>>>>> tends to fail unfortunately.  But you might like to try and we'll see
>>>>>> how it might turn out.
>>>>>
>>>>> I'll let you know how this goes. This is currently taking the most time
>>>>> for the packaging as opengl seems to be a real pain from past
>>>>> experience. Probably the VM factor doesn't help it as maybe there is
>>>>> some more abstraction. Assuming autopkgtest is also ran on debian
>>>>> servers, I wonder if this issue is still present.
>>>>>
>>>>> Kind regards,
>>>>> Shayan Doust
>>>>>
>>>>> On 28/08/2019 12:33, Andreas Tille wrote:
>>>>>> Hi Shayan,
>>>>>>
>>>>>> On Tue, Aug 27, 2019 at 03:40:04AM +0100, Shayan Doust wrote:
>>>>>>> Hello Andreas,
>>>>>>>
>>>>>>> Just to update you on what has been going on. Please disregard my
>>>>>>> previous email to save you time from writing regarding something redundant.
>>>>>>
>>>>>> Ahhh, that really saves time.  I had an extremely busy weekend -
>>>>>> otherwise I would have answered before.
>>>>>>  
>>>>>>> I have added some more overrides and updated the symbol file again to
>>>>>>> which the package is now being built successfully :).
>>>>>>
>>>>>> The symbols file changes are really strange.  I was running again in a
>>>>>> diff.  I have not commited my changes yet but for the moment I have
>>>>>> deactivated the symbols file since it seems we are playing diff - patch
>>>>>> ping-pong for no good reason.
>>>>>>  
>>>>>>> A few errors persist via Lintian when I first built. The Lintian errors
>>>>>>> are "arch-independent-package-contains-binary-or-object" referencing
>>>>>>> usr/lib/x86_64-linux-gnu/libFAST.a and
>>>>>>> "triplet-dir-and-architecture-mismatch" referencing
>>>>>>> usr/lib/x86_64-linux-gnu.
>>>>>>>
>>>>>>> For arch-independent-package-contains-binary-or-object: I have changed
>>>>>>> the architecture of libfast-dev from "all" to "any".
>>>>>>
>>>>>> That's definitely correct - architecture should be any.
>>>>>>  
>>>>>>> non-binmutable-all-depends-any: For the depends field, I have used
>>>>>>> "libfast0 (>= ${source:Version}), libfast0 (<< ${source:Version}.1~)"
>>>>>>> instead of "libfast0 (= ${binary:Version})". Please advise if this
>>>>>>> should be something else.
>>>>>>
>>>>>> I admit I have never seen this.  D-shlibs enforces
>>>>>>     (= ${binary:Version})
>>>>>> and this should be there.  Your.  May be your change was needed once
>>>>>> the package was arch: all ?
>>>>>>  
>>>>>>> I have modified the --movedev option of d-shlibmove as we do not need
>>>>>>> the CL include directory as this will clash with OpenCL's headers, and
>>>>>>> the patch I put erradicates the need for this additional FAST's CL
>>>>>>> directory anyways.
>>>>>>
>>>>>> Sounds sensible even if I can not judge about this since I do not
>>>>>> know the internals of this program.
>>>>>>
>>>>>>> I will try run autopkgtest to make sure the patch has
>>>>>>> not missed anything out although I am personally having opengl driver
>>>>>>> issues that is stopping me from doing this for now.
>>>>>>
>>>>>> Testing with opengl is always a nuisance.  Even with emulation this
>>>>>> tends to fail unfortunately.  But you might like to try and we'll see
>>>>>> how it might turn out.
>>>>>>
>>>>>>> I've written this email in time order while I have been applying some
>>>>>>> commits, so as of right now I do not see anymore Lintian errors or
>>>>>>> warnings. In the meantime, I've also spent the time going through any
>>>>>>> devel documentation I could find with regards to policy and guides.
>>>>>>
>>>>>> I've started a build at home and will send further comments once I'm
>>>>>> home again.
>>>>>>
>>>>>> Kind regards
>>>>>>
>>>>>>      Andreas.
>>>>>>
>>>>>>> On 24/08/2019 18:32, Shayan Doust wrote:
>>>>>>>> Hello Andreas,
>>>>>>>>
>>>>>>>> Just to extend the previous email I sent.
>>>>>>>>
>>>>>>>> I have corrupted my VM which means I cannot get the exact error message
>>>>>>>> however off the top of my head, it seems like d-shlib is failing.
>>>>>>>>
>>>>>>>> The error message consists of many lines stating "There is no package
>>>>>>>> matching [<some qt libraries>] and noone provides it, plese report but
>>>>>>>> to d-shlibs maintainer".
>>>>>>>>
>>>>>>>> I am sure you can replicate this. Does this look like a reporable issue
>>>>>>>> or is this related to perhaps some flags you set for d-shlibs?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Shayan Doust
>>>>>>>>
>>>>>>>> On 23/08/2019 21:28, Shayan Doust wrote:
>>>>>>>>> Hello Andreas,
>>>>>>>>>
>>>>>>>>>> Something which is really strange is that dh_makeshlibs created a diff
>>>>>>>>>> for the symbols file.  I applied that diff and started a rebuild - no
>>>>>>>>>> idea why this is happening.
>>>>>>>>>
>>>>>>>>> I've seen this happen a few times and I've ended up rewriting the
>>>>>>>>> existing symbols file with the new one generated as DEBIAN/symbols.
>>>>>>>>>
>>>>>>>>>> I also switched the packaging to d-shlibs a I promised.  The good thing
>>>>>>>>>> is that it enforces library packaging policy and thus I was able to spot
>>>>>>>>>> missing "Section" fields in the control file.
>>>>>>>>>
>>>>>>>>> Thanks. It makes sense. Admittingly, the thing I've had issues with was
>>>>>>>>> the override option as at the time, did not make sense to me. I'll now
>>>>>>>>> use d-shlibs within future library packages as it does most of the job
>>>>>>>>> itself in a safe way.
>>>>>>>>>
>>>>>>>>>> I'll test this later.
>>>>>>>>>
>>>>>>>>> Thanks. It could just be that d-shlib rectifies this as maybe I was
>>>>>>>>> installing the wrong thing. I will try initiate a build as I have not
>>>>>>>>> done so yet.
>>>>>>>>>
>>>>>>>>>> I promised support - specifically when you are tackling such a huge
>>>>>>>>>> beast. ;-)
>>>>>>>>>
>>>>>>>>> Grateful for this :). It makes more sense visualising the current issues
>>>>>>>>> and how you would rectify it, which I can apply to future packaging.
>>>>>>>>>
>>>>>>>>> Kind Regards,
>>>>>>>>> Shayan Doust
>>>>>>>>>
>>>>>>>>> On 23/08/2019 08:36, Andreas Tille wrote:
>>>>>>>>>> Hi Shayan,
>>>>>>>>>>
>>>>>>>>>> On Thu, Aug 22, 2019 at 09:09:29PM +0100, Shayan Doust wrote:
>>>>>>>>>>> Just to extend the previous email.
>>>>>>>>>>>
>>>>>>>>>>> I created another fresh environment and even used pbuilder and the
>>>>>>>>>>> results are the same, that the build succeeds.
>>>>>>>>>>
>>>>>>>>>> Something which is really strange is that dh_makeshlibs created a diff
>>>>>>>>>> for the symbols file.  I applied that diff and started a rebuild - no
>>>>>>>>>> idea why this is happening.
>>>>>>>>>>
>>>>>>>>>> I also switched the packaging to d-shlibs a I promised.  The good thing
>>>>>>>>>> is that it enforces library packaging policy and thus I was able to spot
>>>>>>>>>> missing "Section" fields in the control file.
>>>>>>>>>>
>>>>>>>>>> Warning: I've commited despite my build did not finished but I wanted
>>>>>>>>>> to push the d-shlibs change soon to get you informed.
>>>>>>>>>>  
>>>>>>>>>>> Additionally, I have attempted to disable RPATH during the build but the
>>>>>>>>>>> error and RPATH still exist, so I am not too sure. If a solution isn't
>>>>>>>>>>> reached, I will write to debian mentors like you have mentioned. I'm not
>>>>>>>>>>> sure if I can just use dpkg-shlibdeps -l<lib> for each executable but
>>>>>>>>>>> this seems like a fair bit of pain to do for all the libraries and
>>>>>>>>>>> executables that exist and are compiled.
>>>>>>>>>>
>>>>>>>>>> I'll test this later.
>>>>>>>>>>
>>>>>>>>>>> Many thanks for looking into this & regards,
>>>>>>>>>>
>>>>>>>>>> I promised support - specifically when you are tackling such a huge
>>>>>>>>>> beast. ;-)
>>>>>>>>>>
>>>>>>>>>> Kind regards
>>>>>>>>>>
>>>>>>>>>>       Andreas.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>>
>>
> 
> 
> 
> 

Attachment: signature.asc
Description: OpenPGP digital signature


Reply to: