Re: Third party code inside Ugene
Hi Yuliya,
On Tue, Feb 05, 2019 at 10:35:53AM +0700, Yuliya Algaer wrote:
>
> 1. git clone https://github.com/ugeneunipro/ugene.git ugene_deb
>
> 2. removed zlib from ~/ugene_deb/src/libs_3rd_party
>
> 3. export UGENE_USE_BUNDLED_ZLIB=0
Ahh, sorry, I stupidly used UGENE_USE_BUNDLED_ZLIB=1.
> Could you please try to do the same steps?
I can confirm that even the packaging (which in principle is what
you suggested, works now without zlib.
Do you have any comment to the licensing issue I mentioned in my
other mail[1] about
Files: src/corelibs/U2Formats/src/ABIFormat.cpp
src/corelibs/U2Formats/src/SCFFormat.cpp
> Thank you,
Thanks again to you
Andreas.
[1] https://lists.debian.org/debian-med/2019/01/msg00152.html
> On 02/01/2019 06:10 PM, Andreas Tille wrote:
> > Hi again,
> >
> > I took the time and re-added
> >
> > src/libs_3rdparty/zlib/zlib.pr[io]
> >
> > and tried to set the variable below. Unfortunately it does not work with
> > removed zlib code:
> >
> > ...
> > cd src/libs_3rdparty/zlib/ && ( test -e Makefile || /usr/lib/qt5/bin/qmake -o Makefile /build/ugene-1.31.1+dfsg1/src/libs_3rdparty/zlib/zlib.pro 'QMAKE_CFLAGS_RELEASE=-g -O2 -fdebug-prefix-map=/build/ugene-1.31.1+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' 'QMAKE_CFLAGS_DEBUG=-g -O2 -fdebug-prefix-map=/build/ugene-1.31.1+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' 'QMAKE_CXXFLAGS_RELEASE=-g -O2 -fdebug-prefix-map=/build/ugene-1.31.1+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' 'QMAKE_CXXFLAGS_DEBUG=-g -O2 -fdebug-prefix-map=/build/ugene-1.31.1+dfsg1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' 'QMAKE_LFLAGS_RELEASE=-Wl,-z,relro -Wl,-z,now' 'QMAKE_LFLAGS_DEBUG=-Wl,-z,relro -Wl,-z,now' QMAKE_STRIP=: PREFIX=/usr QMAKE_CFLAGS_ISYSTEM= QMAKE_CXXFLAGS_ISYSTEM= UGENE_WITHOUT_NON_FREE=1 UGENE_LRELEASE=lrelease-qt5 UGENE_LUPDATE=lupdate-qt5 UGENE_USE_BUNDLED_ZLIB=1 ) && make -f Makefile
> > WARNING: Failure to find: src/adler32.c
> > WARNING: Failure to find: src/compress.c
> > WARNING: Failure to find: src/crc32.c
> > WARNING: Failure to find: src/deflate.c
> > WARNING: Failure to find: src/gzio.c
> > WARNING: Failure to find: src/infback.c
> > WARNING: Failure to find: src/inffast.c
> > WARNING: Failure to find: src/inflate.c
> > WARNING: Failure to find: src/inftrees.c
> > WARNING: Failure to find: src/minigzip.c
> > WARNING: Failure to find: src/trees.c
> > WARNING: Failure to find: src/uncompr.c
> > WARNING: Failure to find: src/zutil.c
> > WARNING: Failure to find: src/crc32.h
> > WARNING: Failure to find: src/deflate.h
> > WARNING: Failure to find: src/inffast.h
> > WARNING: Failure to find: src/inffixed.h
> > WARNING: Failure to find: src/inflate.h
> > WARNING: Failure to find: src/inftrees.h
> > WARNING: Failure to find: src/trees.h
> > WARNING: Failure to find: src/zconf.h
> > WARNING: Failure to find: src/zlib.h
> > WARNING: Failure to find: src/zutil.h
> > WARNING: Failure to find: src/adler32.c
> > WARNING: Failure to find: src/compress.c
> > WARNING: Failure to find: src/crc32.c
> > WARNING: Failure to find: src/deflate.c
> > WARNING: Failure to find: src/gzio.c
> > WARNING: Failure to find: src/infback.c
> > WARNING: Failure to find: src/inffast.c
> > WARNING: Failure to find: src/inflate.c
> > WARNING: Failure to find: src/inftrees.c
> > WARNING: Failure to find: src/minigzip.c
> > WARNING: Failure to find: src/trees.c
> > WARNING: Failure to find: src/uncompr.c
> > WARNING: Failure to find: src/zutil.c
> > WARNING: Failure to find: src/crc32.h
> > WARNING: Failure to find: src/deflate.h
> > WARNING: Failure to find: src/inffast.h
> > WARNING: Failure to find: src/inffixed.h
> > WARNING: Failure to find: src/inflate.h
> > WARNING: Failure to find: src/inftrees.h
> > WARNING: Failure to find: src/trees.h
> > WARNING: Failure to find: src/zconf.h
> > WARNING: Failure to find: src/zlib.h
> > WARNING: Failure to find: src/zutil.h
> > make[2]: Entering directory '/build/ugene-1.31.1+dfsg1/src/libs_3rdparty/zlib'
> > make -f Makefile.Release
> > make[3]: Entering directory '/build/ugene-1.31.1+dfsg1/src/libs_3rdparty/zlib'
> > make[3]: *** No rule to make target 'src/adler32.c', needed by '_tmp/obj/release/adler32.o'. Stop.
> > make[3]: Leaving directory '/build/ugene-1.31.1+dfsg1/src/libs_3rdparty/zlib'
> > make[2]: *** [Makefile:40: release] Error 2
> > ...
> >
> > Kind regards
> >
> > Andreas.
> >
> >
> > On Fri, Feb 01, 2019 at 08:15:29AM +0700, Yuliya Algaer wrote:
> > > Hi Andreas,
> > >
> > > Lets start with zlib first: it is easier that way we do not use any custom
> > > flags, renaming and even have initial support to use system library first.
> > >
> > > Check this code from ugene_globals.pri:
> > >
> > > # By default, UGENE uses bundled zlib on Windows (libs_3rdparty/zlib) and OS
> > > version on Linux.
> > > # To use bundled version on any platform set UGENE_USE_BUNDLED_ZLIB = 1
> > >
> > > defineTest( use_bundled_zlib ) {
> > > contains( UGENE_USE_BUNDLED_ZLIB, 1 ) : return (true)
> > > contains( UGENE_USE_BUNDLED_ZLIB, 0 ) : return (false)
> > > win32 {
> > > return (true)
> > > }
> > > return (false)
> > > }
> > >
> > >
> > > This way you can switch to the system library: bundled sources will not be
> > > used at all.
> > >
> > > So the next steps for Debian packaging could be:
> > >
> > > 1) Switch from CMake to qmake build. This is how we build UGENE for
> > > production.
> > >
> > > 2) Try to build it with no bundled zlib. If it works you can completely
> > > remove zlib folder from UGENE sources for Debian and add zlib-dev as the
> > > required dependency.
> > >
> > >
> > > About Phylip: this is plugin, so it can easily removed from the build &
> > > source code. While most of the 3rd party tools are run from UGENE as
> > > external processes, some of them, and Phylip is the one of them, are
> > > integrated on the source level. Examples of such plugins are: Phylip,
> > > Muscle, HMMER2, KAlign. Unfortunately we can't replace them with pre-built
> > > originals because we integrated them on the source level (modified them).
> > > The good news is that there are only 4-5 of such plugins, while 90% (like
> > > BLAST, HMMER3...) are run as external programs.
> > >
> > >
> > > Please let me know if your experiment with zlib is successful. If yes, we
> > > can try the same with SQLite.
> > >
> > >
> > > Kind regards,
> > >
> > > Yuliya
> > >
> > >
> > >
> > > On 01/31/2019 02:08 PM, Andreas Tille wrote:
> > > > Hi Yuliya,
> > > >
> > > > thanks again for the very productive discussion.
> > > >
> > > > On Thu, Jan 31, 2019 at 08:58:47AM +0700, Yuliya Algaer wrote:
> > > > > > -lugenedbd and -lugenedb: what is the difference between these two?
> > > > > The first one is debug build of the library. The second one is release. All
> > > > > libraries in UGENE are built this way.
> > > > OK, that's very helpful. In Debian by default code will be build
> > > > including debug information and distributed in a separate package.
> > > >
> > > > > > I would recommend to check for the existence of zlib on the system that
> > > > > > builds Ugene and if it exists use the zlib from the system.
> > > > > We can add this check into the build script. But in either case we have to
> > > > > have zlib sources in the bundle UGENE to use it as a fallback.
> > > > Yes, for sure you can do this. For the packaging it is just extremely
> > > > helpful that we can link to the system libraries without needing to
> > > > worry about copies in your code. This saves us from creating patches
> > > > and adapting these for every new upstream version. Finally this kind of
> > > > friction is the reason why we are lagging behind your latest version.
> > > >
> > > > > Another alternative here is to re-check if some compatible form of zlib is
> > > > > available via Qt library. I will check it by myself. Thank you for the hint.
> > > > That's another option but I know to less about Qt to comment on this.
> > > > > > In case the system has an up to date library there is no need to make use
> > > > > > of private copies.
> > > > > Same issue as with zlib here. If there is no library on the system we have
> > > > > to use the one from our sources or ask a user to provide one. UGENE builds
> > > > > on many platforms. On Windows, for example, there are no SQLite by default
> > > > > at all.
> > > > That's fine in the same way. We have no problem if the code copy exists
> > > > since we can even strip it from the source archive without worrying
> > > > whether this might influence the functionality.
> > > >
> > > > > > I'm not sure whether phylip is the only example
> > > > > Phylip is a plugin and can easily be excluded from the build.
> > > > That's interesting. How can we make sure that the Debian packaged
> > > > phylip is used as plugin? We really want to provide the full
> > > > functionality but I think to do so we have to understand this plugin
> > > > system better.
> > > >
> > > > > The only real
> > > > > UGENE dependencies are QT5, zlib & SQLite. If you do not mind that we carry
> > > > > the last 2 in our sources, but automatically detect and use system ones on
> > > > > Debian (since we know they are compatible) this task is not very complex and
> > > > > I will be glad to help you here.
> > > > This would be really great and we would appreciate this very much since
> > > > it would drastically reduce our work.
> > > > > > By using cmake features you can easily check whether a user has these
> > > > > > libraries installed
> > > > > We have only unofficial experimental build with CMake and have no active
> > > > > plans on it. So we have to use qmake here.
> > > > Fine with me - whatever you might implement. While the Debian packaging
> > > > is currently using your CMake files this is probably not a good idea if
> > > > it is only experimental. We can adapt to whatever build system you are
> > > > using (even if my personal observation from lots of software is that
> > > > several projects moved from qmake to cmake. I can not compare these
> > > > systems since I only know these as "consumer" but those upstreams who
> > > > switched seem to have their reasons.
> > > > > Thank you,
> > > > > Yuliya.
> > > > Thanks a lot to you
> > > >
> > > > Andreas.
> > > >
> > >
>
>
>
--
http://fam-tille.de
Reply to: