Re: openfst 1.5.2 high memory usage at compile time
* Giulio Paci <giuliopaci@gmail.com>, 2016-04-18, 16:16:
I noticed that openfst 1.5.2-1 [0] is failing to build on mips and
mipsel architectures [1].
You already pointed out that algo_test.cc compilation requires more
than 2GB on i386, and we already set some reasonable limits for
parallel builds to cope with several possible memory issues.
My understanding of the failure is that there is a 2GB limit per
process on mips architecture. Is it right?
I believe this is the case, yes. (But IANA MIPS expert.)
algo_test.cc is used only in the test suite, so I wonder if we can just
workaround the issue by disabling optimization and using "-O0" for that
specific file. Is this a possible workaround?
I am thinking about adding something like this to rules file:
ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
common-build-arch::
+$(DEB_MAKE_INVOKE) CXXFLAGS="$(CXXFLAGS:-O%=-O0) -std=c++11" -C src/test algo_test.o
endif
I tested this on a mips porterbox. Compling algo_test.o with -O0 does
help, but later linking fails with:
algo_test.o: In function `fst::LookAheadCompose(fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > const&, fst::Fst<fst::ArcTpl<fst::TropicalWeightTpl<float> > > const&, fst::MutableFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*)':
/home/jwilk/openfst-1.5.2/src/test/./algo_test.h:54:(.text+0x304): relocation truncated to fit: R_MIPS_GOT16 against `__stack_chk_guard@@GLIBC_2.4'
/home/jwilk/openfst-1.5.2/src/test/./algo_test.h:67:(.text+0x4e8): relocation truncated to fit: R_MIPS_GOT16 against `__stack_chk_guard@@GLIBC_2.4'
/home/jwilk/openfst-1.5.2/src/test/./algo_test.h:55:(.text+0x588): relocation truncated to fit: R_MIPS_CALL16 against `_Unwind_Resume@@GCC_3.0'
/home/jwilk/openfst-1.5.2/src/test/./algo_test.h:67:(.text+0x598): relocation truncated to fit: R_MIPS_CALL16 against `__stack_chk_fail@@GLIBC_2.4'
algo_test.o: In function `main':
/home/jwilk/openfst-1.5.2/src/test/algo_test.cc:51:(.text+0x5f4): relocation truncated to fit: R_MIPS_GOT16 against `__stack_chk_guard@@GLIBC_2.4'
/home/jwilk/openfst-1.5.2/src/test/algo_test.cc:52:(.text+0x600): relocation truncated to fit: R_MIPS_GOT16 against `FLAGS_fst_verify_properties'
/home/jwilk/openfst-1.5.2/src/test/algo_test.cc:53:(.text+0x60c): relocation truncated to fit: R_MIPS_GOT16 against `FailedNewHandler()'
/home/jwilk/openfst-1.5.2/src/test/algo_test.cc:53:(.text+0x610): relocation truncated to fit: R_MIPS_CALL16 against `std::set_new_handler(void (*)())@@GLIBCXX_3.4'
/home/jwilk/openfst-1.5.2/src/test/algo_test.cc:54:(.text+0x64c): relocation truncated to fit: R_MIPS_CALL16 against `SetFlags(char const*, int*, char***, bool, char const*)'
/home/jwilk/openfst-1.5.2/src/test/algo_test.cc:58:(.text+0x674): relocation truncated to fit: R_MIPS_CALL16 against `time@@GLIBC_2.0'
/home/jwilk/openfst-1.5.2/src/test/algo_test.cc:59:(.text+0x6a4): additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status
GCC manpage says that you should use the -mxgot if you run into such
error. I guess disabling PIE for this test would do the trick too.
Anyway, given that OpenFST is in experimental only, I propose not to
worry about it too much.
In the meanwhile I also asked upstream if it is possible to simplify
that code, in order to reduce its memory requirements.
As a user of a (non-MIPS) machine with only 2 GB of RAM, I'd appreciate
slimming it down!
--
Jakub Wilk
Reply to: