Re: Help needed to update to latest fastqc
Hi,
After a fair amount of struggling I conclude that I do not have all the
code required to build this thing from source.
The distribution of JHDF5 from
https://wiki-bsse.ethz.ch/pages/viewpage.action?pageId=26609113
contains some Java source code and a bunch of pre-compiled JNI shared
libs, eg.:
sis-jhdf5/lib/native/jhdf5/amd64-Linux/libjhdf5.so
The source for this file is not to be found anywhere that I can see it.
One might assume (and the FAQ implies) that the file is equivalent
to /usr/lib/jni/libjhdf5.so provided by the exiting package libjhdf5-jni
but it isn't! Inspecting it with readelf shows it is full of symbols in
the ETH SIS namespace:
tbooth@balisaur[sis-jhdf5] readelf -sW ./sis-jhdf5/lib/native/jhdf5/amd64-Linux/libjhdf5.so | grep ch_systemsx | head
19: 0000000000039a40 58 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Scopy
31: 000000000002e930 181 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Dchdir_1ext
35: 000000000003b590 918 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Sget_1select_1bounds
40: 00000000000387f0 45 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Pcreate_1xfer_1abort
41: 0000000000038820 66 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Pget_1char_1encoding
46: 0000000000034500 203 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Oopen
47: 0000000000034ae0 363 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Pget_1version
52: 0000000000035210 66 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Pset_1layout
65: 000000000003c560 126 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Tget_1member_1name
68: 0000000000032da0 58 FUNC GLOBAL DEFAULT 10 Java_ch_systemsx_cisd_hdf5_hdf5lib_H5_H5Idec_1ref
...and many more...
So, this .so file is definitely specific to this library. It seems to
be a superset of a fork of the upstream NCSA code. The only contact
details I can find for the library are for Dr. Bernd Rinn who is the
head of the whole division, but I shall contact him personally and see
if he can pass the query on.
If Bernd can't help, then unless I've missed a trick we will have to
consider that this library and anything that depends on it (eg FastQC)
are not fully open source :-(
Cheers,
TIM
On Wed, 2015-08-12 at 15:00 +0200, Andreas Tille wrote:
> Hi Tim,
>
> from my *very* naive point of view your considerations make sense.
> Olivier might have some more educated opinion (and perhaps some code?).
>
> I'll be available vor sponsering as usual.
>
> Thanks for your work on this
>
> Andreas.
>
> On Wed, Aug 12, 2015 at 11:44:19AM +0100, Tim Booth wrote:
> > Hi Andreas and Olivier,
> >
> > I'm going to have a crack at the FastQC update. I've been poking at the
> > HDF5 packages and here are my conclusions so far:
> >
> > ---
> >
> > We have 2 Free Java libraries for HDF5:
> >
> > 1) The "NCSA" lib distributed by hdfgroup.org with classes in namespace
> > ncsa.hdf.hdf5lib, packaged on Debian as libjhdf5-java.
> >
> > 2) The "CISD" aka. "SIS" lib distributed by ethz.ch with classes in the
> > namespace ch.systemsx, not currently packaged on Debian.
> >
> > The SIS library forks some Java code from the ncsa.hdf namespace, and
> > also needs to link against the NCSA native library (via JNI) at runtime.
> >
> > So, trying to package sis-jhdf5:
> >
> > wget
> > 'https://wiki-bsse.ethz.ch/download/attachments/26609237/sis-jhdf5-14.12.1-r33502.zip?version=1&modificationDate=1424599261225&api=v2'
> > # The source is in a zip within the zip...
> > unzip sis-*
> > mkdir sis-jhdf5-14.12.1
> > cd !$
> > unzip ../sis-jhdf5/src/sis-jhdf5-src.zip
> >
> > For compilation to succeed, we also need sis-base, not to be confused
> > with the entirely separate project at sis.apache.org!
> >
> > wget
> > http://bs-svn01.ethz.ch/repos/cisd/libraries/trunk/sis-base/sis-base-src.zip
> >
> > The code also depends on their own args4j library, but only for the CLI
> > under ch/systemsx/cisd/hdf5/h5ar/HDF5ArchiverMain.java which we can
> > simply leave out just now.
> >
> > So - can we exclude the ncsa.hdf.hdf5lib.* classes from sis-jhdf5 and
> > compile it against the existing classes in /usr/share/java/jhdf5.jar?
> > My conclusion is not, because...
> >
> > The version of ncsa/hdf/hdf5lib/exceptions/HDF5Exception.java supplied
> > with the sis-jhdf5 code subclasses RuntimeException but the version in
> > libhdf5-java does not. The result is that the code here does not need
> > to declare the exceptions being thrown, and thus simply ignores them
> > all. One can go through and add all "throws" declarations (there are a
> > _lot_ of them!!), but then any client code using the library will also
> > break unless it is likewise patched or the code is modified to handle
> > the exceptions internally. I think we'd best just admit defeat here and
> > use the code supplied with sis-jhdf5.
> >
> > So, I'm going to try rolling two new packages, and see if that allows me
> > to compile the latest FastQC:
> >
> > libsis-base-java (should be simple)
> > libsis-jhdf5-java (depends on libsis-base-java + existing libjhdf5-jni)
> >
> > I'll report back on progress and commit to SVN as usual. Let me know if
> > you think I'm missing something.
> >
> > Cheers,
> >
> > TIM
> >
> >
> > On Thu, 2015-06-25 at 09:37 +0200, Andreas Tille wrote:
> > > Hi Olivier,
> > >
> > > any news about this hdf5lib packaging?
> > >
> > > Kind regards
> > >
> > > Andreas.
> > >
> > > On Tue, May 05, 2015 at 09:25:23AM +0200, Olivier Sallou wrote:
> > > >
> > > >
> > > > On 05/04/2015 10:52 PM, Andreas Tille wrote:
> > > > > Hi Olivier,
> > > > >
> > > > > On Mon, May 04, 2015 at 06:41:10PM +0200, Olivier Sallou wrote:
> > > > >> File fastqc/uk/ac/babraham/FastQC/Sequence/Fast5File.java imports
> > > > >> HDFS5Factory from ch.systemsx.cisd.hdf5
> > > > >>
> > > > >> but jhdf5.jar only contains files like: ncsa/hdf/hdf5lib/... and no
> > > > >> HDF5Factory
> > > > >>
> > > > >> this jhdf5 is not the correctl lib (or an other one is needed).
> > > > > I guess its this one:
> > > > >
> > > > > https://svncisd.ethz.ch/doc/hdf5/hdf5-8.10/ch/systemsx/cisd/hdf5/package-summary.html
> > > > seems that original code source is the same
> > > > (https://svncisd.ethz.ch/repos/cisd/jhdf5/trunk/source/java/)
> > > > (/ncsa/...) but that /ch/... has not been included for a reason
> > > > >
> > > > >> In addition, in debian/patches/build.xml is missing commons-math3.jar to
> > > > >> be added in pathelement for classpath.
> > > > >>
> > > > >> for duplicate classes, I don't yet, but hdf5 first need to be resolved.
> > > > > If I do not hear from you I'll try the URL above tomorrow (if nobody
> > > > > else might beat me which would for sure welcome as always).
> > > > >
> > > > > Kind regards
> > > > >
> > > > > Andreas.
> > > > >
> > > >
> > > > --
> > > > Olivier Sallou
> > > > IRISA / University of Rennes 1
> > > > Campus de Beaulieu, 35000 RENNES - FRANCE
> > > > Tel: 02.99.84.71.95
> > > >
> > > > gpg key id: 4096R/326D8438 (keyring.debian.org)
> > > > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
> > > >
> > > >
> > > > --
> > > > To UNSUBSCRIBE, email to debian-med-REQUEST@lists.debian.org
> > > > with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> > > > Archive: https://lists.debian.org/55487063.1090804@irisa.fr
> > > >
> > > >
> > >
> > > --
> > > http://fam-tille.de
> > >
> > >
> >
> > --
> > Nothing is inherently mysterious - nothing that actually exists, that
> > is. If I am ignorant about a phenomenon, that is a fact about my state
> > of mind, not a fact about the phenomenon; to worship a phenomenon
> > because it seems so wonderfully mysterious, is to worship your own
> > ignorance.
> >
> > - Eliezer Yudkowsky
> >
> >
>
--
Nothing is inherently mysterious - nothing that actually exists, that
is. If I am ignorant about a phenomenon, that is a fact about my state
of mind, not a fact about the phenomenon; to worship a phenomenon
because it seems so wonderfully mysterious, is to worship your own
ignorance.
- Eliezer Yudkowsky
Reply to: