Re: [MoM] FAST progress (was Re: [MoM] fast: Add further dependencies to enable chroot / cowbuilder to build)
Hi Shayan,
On Fri, Aug 30, 2019 at 08:46:15PM +0100, Shayan Doust wrote:
>
> Need a suggestion for a directory because something is slightly unclear.
> I am using the contents in doc/fonts and doc/images and copying these
> over to /usr/share/doc/FAST/assets. FAST is designed to dynamically look
> for this via <current library (*.so) location>/../share/doc/FAST. This
> is dynamic because these assets are needed both at compilation time, and
> during runtime and the build environment will not have these installed
> statically in /usr/share/doc/FAST/assets.
This does not sound as if it would be that these files are helpful as
user documentation.
> The only time I see these assets ever coming to use are the examples and
> the test suite itself (FAST likes to set and display its own font from
> the font directory and images from the image directory). In a sense,
> this is needed for the functionality however on the other hand, these
> assets seem to be useful from the example code on the wiki and could
> maybe still be classed as documentation in itself.
>
> Should I keep using /usr/share/doc/FAST/assets or should I instead use
> /usr/lib/FAST in cases like these.
To my gut feeling the most appropriate location for these files would be
/usr/share/fast
since /usr/lib is for architecture dependant files and these seem to be
architecture independant. However, if it might involve heavy patching
even /usr/lib is fine.
> Thanks for your time,
You are welcome as always
Andreas.
> On 30/08/2019 17:26, Shayan Doust wrote:
> > Hello Andreas,
> >
> >> Unable to open directory /usr/../doc//fonts
> >
> > That's very annoying. The docs file is expected to be installed in /usr
> > and contains "FAST" image branding and some other assets that are used
> > in the program. I will see if FAST allows for dynamic pathing and where
> > it actually wants me to install the doc folder. The reason I did not
> > install the doc folder under /usr/share and excluded it for now is
> > because it serves no purpose to the end user. So now I will look into it
> > and if not, introduce another patch to cater for the doc directory to be
> > under a different location.
> >
> > SIGSEVG probably because it expects these "doc" assets. Luckily there is
> > no early assertion of CL which means the CL patch probably worked. I
> > will try offloading this onto my build server, and manually testing the
> > test suite on another spare laptop and hopefully opencl is supported.
> >
> >> A general remark: The package libfast-examples contains compiled
> >> executables in /usr/lib/fast. Users usually expect examples in
> >>
> >> /usr/share/doc/PKGNAME/examples
> >>
> >> (that's where dh_installexamples puts files). Usually these are not
> >> binaries. It depends a bit what we want to teach users by these
> >> examples. If it is how to develop with libfast-dev than it would be
> >> better to provide the source code files including a Makefile to compile
> >> these examples (and may be change something and try what's happening).
> >>
> >> If something else should be demonstrated and these executables might
> >> have some extra value I'd consider it necessary to add some information
> >> under /usr/share/doc/libfast-examples/examples or similar to inform
> >> users where they can find those files and what to do.
> >
> > Ahh I see. I am still fairly uninformed as to what these examples
> > represent (until I rectify this opencl issue). I thought these binaries
> > would be nice because they represent the examples on the wiki[1].
> >
> >> If it is how to develop with libfast-dev than it would be
> >> better to provide the source code files including a Makefile to
> >> compile these examples (and may be change something and try what's
> >> happening).
> >
> > That could work, although that would mean writing the examples and
> > writing my own makefile which could work as this is not readily
> > available, however I think a simpler option would be the second point
> > you suggested. Maybe adding a documentation / some file under
> > /usr/share/doc/libfast-examples/examples to both explain the purpose,
> > refer to the wiki and the new example location. I will play with these
> > ideas once I figure out this opencl.
> >
> > Kind regards & thanks for your help,
> > Shayan Doust
> >
> > [1]: https://github.com/smistad/FAST/wiki/Examples
> >
> > On 30/08/2019 17:05, Andreas Tille wrote:
> >> Hi Shayan,
> >>
> >> please git pull for two commits (library name to make sure fast-examples
> >> can be installed and Standards-Version).
> >>
> >> On Thu, Aug 29, 2019 at 01:22:51AM +0100, Shayan Doust wrote:
> >>> Hello Andreas,
> >>>
> >>>> In libhmsbeagle I had to deal with this in build time tests. May be
> >>>> you check the Build-Depends for some additional inspiration. No idea
> >>>> whether this might help.
> >>>
> >>> Many thanks for your reply. This has made some slight difference. A few
> >>> things to note.
> >>>
> >>> Running clinfo returns my CPU as a platform, however "what():
> >>> clGetPlatformIDs" is still thrown. Running the test binary, I see some
> >>> more information. I am not sure the significance of these as opposed to
> >>> the separate example binaries, however here is a short snippet:
> >>>
> >>> ...
> >>>
> >>> -------------------------------------------------------------------------------
> >>> /home/shayandoust/Desktop/FAST/fast/source/FAST/Tests/Benchmarks.cpp:106
> >>> ...............................................................................
> >>>
> >>> /home/shayandoust/Desktop/FAST/fast/source/FAST/Tests/Benchmarks.cpp:106: FAILED:
> >>> due to unexpected exception with message:
> >>> There was an error while getting OpenCL devices: clGetDeviceIDs
> >>>
> >>> INFO: QApp already exists..
> >>> INFO: Device manager initialize..
> >>> INFO: Initial GLX context 0x5574dc453ae0
> >>> INFO: Found 1 OpenCL platforms.
> >>> INFO: 1 platforms selected for inspection.
> >>> INFO: Platform 0: Mesa
> >>> INFO: This platform has
> >>> -------------------------------------------------------------------------------
> >>> Pipeline C
> >>> -------------------------------------------------------------------------------
> >>> /home/shayandoust/Desktop/FAST/fast/source/FAST/Tests/Benchmarks.cpp:167
> >>> ...............................................................................
> >>>
> >>> /home/shayandoust/Desktop/FAST/fast/source/FAST/Tests/Benchmarks.cpp:167: FAILED:
> >>> due to unexpected exception with message:
> >>> There was an error while getting OpenCL devices: clGetDeviceIDs
> >>>
> >>> ===============================================================================
> >>> test cases: 221 | 32 passed | 189 failed
> >>> assertions: 385 | 196 passed | 189 failed
> >>
> >> I get:
> >>
> >> $ sh run-unit-test
> >> Unpacking test data
> >> Invoking testFAST
> >> INFO: Creating new QApp
> >> WARNING: Unable to open the configuration file /usr/fast_configuration.txt. Using defaults instead.
> >>
> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >> testFAST is a Catch v1.5.9 host application.
> >> Run with -? for options
> >>
> >> -------------------------------------------------------------------------------
> >> Airway segmentation
> >> -------------------------------------------------------------------------------
> >> /build/fast-3.0.0~rc3+ds/source/FAST/Algorithms/AirwaySegmentation/airwaySegmentationTests.cpp:49
> >> ...............................................................................
> >>
> >> /build/fast-3.0.0~rc3+ds/source/FAST/Algorithms/AirwaySegmentation/airwaySegmentationTests.cpp:49: FAILED:
> >> due to unexpected exception with message:
> >> Unable to open directory /usr/../doc//fonts
> >>
> >> INFO: QApp already exists..
> >> INFO: Creating new GL context for computation thread
> >> WARNING: Your system uses comma as decimal point.
> >> WARNING: This will now be changed to dot to avoid any comma related bugs.
> >> INFO: Device manager initialize..
> >> INFO: Initial GLX context 0x563a2ac8d8c0
> >> INFO: Found 1 OpenCL platforms.
> >> INFO: 1 platforms selected for inspection.
> >> INFO: Platform 0: Intel
> >> INFO: This platform has 0 available devices in total
> >> INFO: Looking for GPU devices only.
> >> INFO: 1 selected.
> >> INFO: Inspecting device 0 with the name Intel(R) HD Graphics Haswell Ultrabook GT2 Mobile
> >> INFO: Initial GLX context 0x563a2ac8d8c0
> >> -------------------------------------------------------------------------------
> >> Block matching 2D
> >> -------------------------------------------------------------------------------
> >> /build/fast-3.0.0~rc3+ds/source/FAST/Algorithms/BlockMatching/Tests.cpp:10
> >> ...............................................................................
> >>
> >> /build/fast-3.0.0~rc3+ds/source/FAST/Algorithms/BlockMatching/Tests.cpp:10: FAILED:
> >> due to a fatal error condition:
> >> SIGSEGV - Segmentation violation signal
> >>
> >> ===============================================================================
> >> test cases: 2 | 2 failed
> >> assertions: 2 | 2 failed
> >>
> >>> spent a few hours reading the source and attempting to search up the
> >>> error message, although I'm trying not to put any additional load on
> >>> you. If you do lack time, I could always ask in mentors.
> >>
> >> The missing font is a bit suspicious but I'm absolutely uninformed
> >> about all this graphics stuff to be more helpful here.
> >>
> >>
> >> A general remark: The package libfast-examples contains compiled
> >> executables in /usr/lib/fast. Users usually expect examples in
> >>
> >> /usr/share/doc/PKGNAME/examples
> >>
> >> (that's where dh_installexamples puts files). Usually these are not
> >> binaries. It depends a bit what we want to teach users by these
> >> examples. If it is how to develop with libfast-dev than it would be
> >> better to provide the source code files including a Makefile to compile
> >> these examples (and may be change something and try what's happening).
> >>
> >> If something else should be demonstrated and these executables might
> >> have some extra value I'd consider it necessary to add some information
> >> under /usr/share/doc/libfast-examples/examples or similar to inform
> >> users where they can find those files and what to do.
> >>
> >> Kind regards
> >>
> >> Andreas.
> >>
> >>> Many thanks,
> >>> Shayan Doust
> >>>
> >>> On 28/08/2019 20:04, Andreas Tille wrote:
> >>>> On Wed, Aug 28, 2019 at 06:18:59PM +0100, Shayan Doust wrote:
> >>>>> Just as I anticipated, it looks like it will not work as I use
> >>>>> virtualbox and debian is essentially virtualised. So not only can I not
> >>>>> passthrough the host GPU, I am unable to use any drivers which means
> >>>>> that the error I get even when running any of the fast-examples binaries
> >>>>> (apart from OpenIGTLinkServer) is "what(): clGetPlatformIDs", which is
> >>>>> verified by running "clinfo" which returns 0 as the number of platforms
> >>>>> - it seems like I have hit a dead end unless there is some work around
> >>>>> which I have my doubts in.
> >>>>
> >>>> In libhmsbeagle I had to deal with this in build time tests. May be
> >>>> you check the Build-Depends for some additional inspiration. No idea
> >>>> whether this might help.
> >>>>
> >>>> Kind regards
> >>>>
> >>>> Andreas (offline for the next 20 hours)
> >>>>
> >>>>> Kind regards,
> >>>>> Shayan Doust
> >>>>>
> >>>>> On 28/08/2019 16:33, Shayan Doust wrote:
> >>>>>> Hello Andreas,
> >>>>>>
> >>>>>>> Ahhh, that really saves time. I had an extremely busy weekend -
> >>>>>>> otherwise I would have answered before.
> >>>>>>
> >>>>>> No problem. At least I prevented you from answering an old question :).
> >>>>>>
> >>>>>>> The symbols file changes are really strange. I was running again in a
> >>>>>>> diff. I have not commited my changes yet but for the moment I have
> >>>>>>> deactivated the symbols file since it seems we are playing diff - patch
> >>>>>>> ping-pong for no good reason.
> >>>>>>
> >>>>>> I haven't seen this happen with other libraries so it is strange.
> >>>>>> Perhaps our environments are different in some way? It still shouldn't
> >>>>>> matter though as the library itself is one version and not changing.
> >>>>>>
> >>>>>>> I admit I have never seen this. D-shlibs enforces
> >>>>>>> (= ${binary:Version})
> >>>>>>> and this should be there. Your. May be your change was needed once
> >>>>>>> the package was arch: all ?
> >>>>>>
> >>>>>> You're correct. I changed the depends before I realised the arch was
> >>>>>> incorrect, and as usual I forgot to backtrack and undo my change to
> >>>>>> depends. I've just pushed this change.
> >>>>>>
> >>>>>>> Testing with opengl is always a nuisance. Even with emulation this
> >>>>>>> tends to fail unfortunately. But you might like to try and we'll see
> >>>>>>> how it might turn out.
> >>>>>>
> >>>>>> I'll let you know how this goes. This is currently taking the most time
> >>>>>> for the packaging as opengl seems to be a real pain from past
> >>>>>> experience. Probably the VM factor doesn't help it as maybe there is
> >>>>>> some more abstraction. Assuming autopkgtest is also ran on debian
> >>>>>> servers, I wonder if this issue is still present.
> >>>>>>
> >>>>>> Kind regards,
> >>>>>> Shayan Doust
> >>>>>>
> >>>>>> On 28/08/2019 12:33, Andreas Tille wrote:
> >>>>>>> Hi Shayan,
> >>>>>>>
> >>>>>>> On Tue, Aug 27, 2019 at 03:40:04AM +0100, Shayan Doust wrote:
> >>>>>>>> Hello Andreas,
> >>>>>>>>
> >>>>>>>> Just to update you on what has been going on. Please disregard my
> >>>>>>>> previous email to save you time from writing regarding something redundant.
> >>>>>>>
> >>>>>>> Ahhh, that really saves time. I had an extremely busy weekend -
> >>>>>>> otherwise I would have answered before.
> >>>>>>>
> >>>>>>>> I have added some more overrides and updated the symbol file again to
> >>>>>>>> which the package is now being built successfully :).
> >>>>>>>
> >>>>>>> The symbols file changes are really strange. I was running again in a
> >>>>>>> diff. I have not commited my changes yet but for the moment I have
> >>>>>>> deactivated the symbols file since it seems we are playing diff - patch
> >>>>>>> ping-pong for no good reason.
> >>>>>>>
> >>>>>>>> A few errors persist via Lintian when I first built. The Lintian errors
> >>>>>>>> are "arch-independent-package-contains-binary-or-object" referencing
> >>>>>>>> usr/lib/x86_64-linux-gnu/libFAST.a and
> >>>>>>>> "triplet-dir-and-architecture-mismatch" referencing
> >>>>>>>> usr/lib/x86_64-linux-gnu.
> >>>>>>>>
> >>>>>>>> For arch-independent-package-contains-binary-or-object: I have changed
> >>>>>>>> the architecture of libfast-dev from "all" to "any".
> >>>>>>>
> >>>>>>> That's definitely correct - architecture should be any.
> >>>>>>>
> >>>>>>>> non-binmutable-all-depends-any: For the depends field, I have used
> >>>>>>>> "libfast0 (>= ${source:Version}), libfast0 (<< ${source:Version}.1~)"
> >>>>>>>> instead of "libfast0 (= ${binary:Version})". Please advise if this
> >>>>>>>> should be something else.
> >>>>>>>
> >>>>>>> I admit I have never seen this. D-shlibs enforces
> >>>>>>> (= ${binary:Version})
> >>>>>>> and this should be there. Your. May be your change was needed once
> >>>>>>> the package was arch: all ?
> >>>>>>>
> >>>>>>>> I have modified the --movedev option of d-shlibmove as we do not need
> >>>>>>>> the CL include directory as this will clash with OpenCL's headers, and
> >>>>>>>> the patch I put erradicates the need for this additional FAST's CL
> >>>>>>>> directory anyways.
> >>>>>>>
> >>>>>>> Sounds sensible even if I can not judge about this since I do not
> >>>>>>> know the internals of this program.
> >>>>>>>
> >>>>>>>> I will try run autopkgtest to make sure the patch has
> >>>>>>>> not missed anything out although I am personally having opengl driver
> >>>>>>>> issues that is stopping me from doing this for now.
> >>>>>>>
> >>>>>>> Testing with opengl is always a nuisance. Even with emulation this
> >>>>>>> tends to fail unfortunately. But you might like to try and we'll see
> >>>>>>> how it might turn out.
> >>>>>>>
> >>>>>>>> I've written this email in time order while I have been applying some
> >>>>>>>> commits, so as of right now I do not see anymore Lintian errors or
> >>>>>>>> warnings. In the meantime, I've also spent the time going through any
> >>>>>>>> devel documentation I could find with regards to policy and guides.
> >>>>>>>
> >>>>>>> I've started a build at home and will send further comments once I'm
> >>>>>>> home again.
> >>>>>>>
> >>>>>>> Kind regards
> >>>>>>>
> >>>>>>> Andreas.
> >>>>>>>
> >>>>>>>> On 24/08/2019 18:32, Shayan Doust wrote:
> >>>>>>>>> Hello Andreas,
> >>>>>>>>>
> >>>>>>>>> Just to extend the previous email I sent.
> >>>>>>>>>
> >>>>>>>>> I have corrupted my VM which means I cannot get the exact error message
> >>>>>>>>> however off the top of my head, it seems like d-shlib is failing.
> >>>>>>>>>
> >>>>>>>>> The error message consists of many lines stating "There is no package
> >>>>>>>>> matching [<some qt libraries>] and noone provides it, plese report but
> >>>>>>>>> to d-shlibs maintainer".
> >>>>>>>>>
> >>>>>>>>> I am sure you can replicate this. Does this look like a reporable issue
> >>>>>>>>> or is this related to perhaps some flags you set for d-shlibs?
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Shayan Doust
> >>>>>>>>>
> >>>>>>>>> On 23/08/2019 21:28, Shayan Doust wrote:
> >>>>>>>>>> Hello Andreas,
> >>>>>>>>>>
> >>>>>>>>>>> Something which is really strange is that dh_makeshlibs created a diff
> >>>>>>>>>>> for the symbols file. I applied that diff and started a rebuild - no
> >>>>>>>>>>> idea why this is happening.
> >>>>>>>>>>
> >>>>>>>>>> I've seen this happen a few times and I've ended up rewriting the
> >>>>>>>>>> existing symbols file with the new one generated as DEBIAN/symbols.
> >>>>>>>>>>
> >>>>>>>>>>> I also switched the packaging to d-shlibs a I promised. The good thing
> >>>>>>>>>>> is that it enforces library packaging policy and thus I was able to spot
> >>>>>>>>>>> missing "Section" fields in the control file.
> >>>>>>>>>>
> >>>>>>>>>> Thanks. It makes sense. Admittingly, the thing I've had issues with was
> >>>>>>>>>> the override option as at the time, did not make sense to me. I'll now
> >>>>>>>>>> use d-shlibs within future library packages as it does most of the job
> >>>>>>>>>> itself in a safe way.
> >>>>>>>>>>
> >>>>>>>>>>> I'll test this later.
> >>>>>>>>>>
> >>>>>>>>>> Thanks. It could just be that d-shlib rectifies this as maybe I was
> >>>>>>>>>> installing the wrong thing. I will try initiate a build as I have not
> >>>>>>>>>> done so yet.
> >>>>>>>>>>
> >>>>>>>>>>> I promised support - specifically when you are tackling such a huge
> >>>>>>>>>>> beast. ;-)
> >>>>>>>>>>
> >>>>>>>>>> Grateful for this :). It makes more sense visualising the current issues
> >>>>>>>>>> and how you would rectify it, which I can apply to future packaging.
> >>>>>>>>>>
> >>>>>>>>>> Kind Regards,
> >>>>>>>>>> Shayan Doust
> >>>>>>>>>>
> >>>>>>>>>> On 23/08/2019 08:36, Andreas Tille wrote:
> >>>>>>>>>>> Hi Shayan,
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Aug 22, 2019 at 09:09:29PM +0100, Shayan Doust wrote:
> >>>>>>>>>>>> Just to extend the previous email.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I created another fresh environment and even used pbuilder and the
> >>>>>>>>>>>> results are the same, that the build succeeds.
> >>>>>>>>>>>
> >>>>>>>>>>> Something which is really strange is that dh_makeshlibs created a diff
> >>>>>>>>>>> for the symbols file. I applied that diff and started a rebuild - no
> >>>>>>>>>>> idea why this is happening.
> >>>>>>>>>>>
> >>>>>>>>>>> I also switched the packaging to d-shlibs a I promised. The good thing
> >>>>>>>>>>> is that it enforces library packaging policy and thus I was able to spot
> >>>>>>>>>>> missing "Section" fields in the control file.
> >>>>>>>>>>>
> >>>>>>>>>>> Warning: I've commited despite my build did not finished but I wanted
> >>>>>>>>>>> to push the d-shlibs change soon to get you informed.
> >>>>>>>>>>>
> >>>>>>>>>>>> Additionally, I have attempted to disable RPATH during the build but the
> >>>>>>>>>>>> error and RPATH still exist, so I am not too sure. If a solution isn't
> >>>>>>>>>>>> reached, I will write to debian mentors like you have mentioned. I'm not
> >>>>>>>>>>>> sure if I can just use dpkg-shlibdeps -l<lib> for each executable but
> >>>>>>>>>>>> this seems like a fair bit of pain to do for all the libraries and
> >>>>>>>>>>>> executables that exist and are compiled.
> >>>>>>>>>>>
> >>>>>>>>>>> I'll test this later.
> >>>>>>>>>>>
> >>>>>>>>>>>> Many thanks for looking into this & regards,
> >>>>>>>>>>>
> >>>>>>>>>>> I promised support - specifically when you are tackling such a huge
> >>>>>>>>>>> beast. ;-)
> >>>>>>>>>>>
> >>>>>>>>>>> Kind regards
> >>>>>>>>>>>
> >>>>>>>>>>> Andreas.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >>
> >
>
--
http://fam-tille.de
Reply to: