mipsel out-of-memory building dolfinx 0.4.0
Hi MIPS porters, I need helping with the build of fenics-dolfinx on
mipsel from experimental,
https://salsa.debian.org/science-team/fenics/fenics-dolfinx/-/tree/experimental
It previously built fine, but the latest version 1:0.4.1 has evidently
grown large enough to break on mipsel.
mip64el builds successfully, as we might expect. But it's not simply a
32-bit issue in the sense that i386, armel, armhf are all still building
successfully.
Build logs can be found at
https://buildd.debian.org/status/logs.php?pkg=fenics-dolfinx&arch=mipsel
In the build for 1:0.4.1-1exp1 we see:
[ 83%] Building CXX object CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o
/usr/bin/c++ -DBOOST_ALL_NO_LIB -DBOOST_CHRONO_DYN_LINK
-DBOOST_TIMER_DYN_LINK -DDOLFINX_VERSION=\"0.4.1\" -DHAS_PTSCOTCH
-DHAS_SLEPC -D_FILE_OFFSET_BITS=64 -Dcpp_EXPORTS
-I/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/lib/python3/dist-packages/petsc4py/include
-I/usr/lib/python3/dist-packages/mpi4py/include -isystem
/usr/include/python3.9 -isystem
/usr/lib/python3/dist-packages/pybind11/include -isystem
/usr/lib/python3/dist-packages/ffcx/codegeneration -isystem
/usr/lib/mipsel-linux-gnu/openmpi/include -isystem
/usr/lib/mipsel-linux-gnu/openmpi/include/openmpi -isystem
/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/include -isystem
/usr/include/hdf5/openmpi -isystem
/usr/lib/slepcdir/slepc3.16/mipsel-linux-gnu-complex/include -g -O2
-ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat
-Werror=format-security -isystem
/<<PKGBUILDDIR>>/debian/tmp-complex/usr/include -DVERSION_INFO=\"0.4.1\"
-O3 -DNDEBUG -fPIC -fvisibility=hidden -Wall -Werror -pedantic -flto
-fno-fat-lto-objects -pthread -Wdate-time -D_FORTIFY_SOURCE=2 -std=c++17
-MD -MT CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o -MF
CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o.d -o
CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o -c
/<<PKGBUILDDIR>>/python/dolfinx/wrappers/nls.cpp
[ 91%] Building CXX object
CMakeFiles/cpp.dir/dolfinx/wrappers/refinement.cpp.o
/usr/bin/c++ -DBOOST_ALL_NO_LIB -DBOOST_CHRONO_DYN_LINK
-DBOOST_TIMER_DYN_LINK -DDOLFINX_VERSION=\"0.4.1\" -DHAS_PTSCOTCH
-DHAS_SLEPC -D_FILE_OFFSET_BITS=64 -Dcpp_EXPORTS
-I/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/lib/python3/dist-packages/petsc4py/include
-I/usr/lib/python3/dist-packages/mpi4py/include -isystem
/usr/include/python3.9 -isystem
/usr/lib/python3/dist-packages/pybind11/include -isystem
/usr/lib/python3/dist-packages/ffcx/codegeneration -isystem
/usr/lib/mipsel-linux-gnu/openmpi/include -isystem
/usr/lib/mipsel-linux-gnu/openmpi/include/openmpi -isystem
/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/include -isystem
/usr/include/hdf5/openmpi -isystem
/usr/lib/slepcdir/slepc3.16/mipsel-linux-gnu-complex/include -g -O2
-ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat
-Werror=format-security -isystem
/<<PKGBUILDDIR>>/debian/tmp-complex/usr/include -DVERSION_INFO=\"0.4.1\"
-O3 -DNDEBUG -fPIC -fvisibility=hidden -Wall -Werror -pedantic -flto
-fno-fat-lto-objects -pthread -Wdate-time -D_FORTIFY_SOURCE=2 -std=c++17
-MD -MT CMakeFiles/cpp.dir/dolfinx/wrappers/refinement.cpp.o -MF
CMakeFiles/cpp.dir/dolfinx/wrappers/refinement.cpp.o.d -o
CMakeFiles/cpp.dir/dolfinx/wrappers/refinement.cpp.o -c
/<<PKGBUILDDIR>>/python/dolfinx/wrappers/refinement.cpp
cc1plus: out of memory allocating 17707045 bytes after a total of
42041344 bytes
In the first instance I tried switching off LTO, adding the cmake
configuration "set(CMAKE_INTERPROCEDURAL_OPTIMIZATION FALSE)" which
deactivates -flto.
But it didn't fix the problem. The corresponding log for 1:0.4.1-1exp2
shows much the same problem,
virtual memory exhausted: Cannot allocate memory
I then tried building on the porterbox eller.debian.org, reducing
optimization from -O3 to -O2, but that gave the same result.
Finally I tried with no optimizations at all (-O0) via cmake option
add_compile_options(-O0). This did allow the object files to build, but
then linking failed with relocation truncated:
[100%] Linking CXX shared module
../lib.linux-mips-3.10/dolfinx/cpp.cpython-310-mipsel-linux-gnu.so
/usr/bin/cmake -E cmake_link_script CMakeFiles/cpp.dir/link.txt
--verbose=1
/usr/bin/c++ -fPIC -g -O2
-ffile-prefix-map=/home/dparsons/dolfinx/fenics-dolfinx=.
-fstack-protector-strong -Wformat -Werror=format-security -isystem
/home/dparsons/dolfinx/fenics-dolfinx/debian/tmp-complex/usr
/include -DVERSION_INFO=\"0.4.1\" -O3 -DNDEBUG -pthread -Wl,-z,relro
-shared -o
../lib.linux-mips-3.10/dolfinx/cpp.cpython-310-mipsel-linux-gnu.so
CMakeFiles/cpp.dir/dolfinx/wrappers/dolfinx.cpp.o CMakeFiles/cp
p.dir/dolfinx/wrappers/common.cpp.o
CMakeFiles/cpp.dir/dolfinx/wrappers/fem.cpp.o
CMakeFiles/cpp.dir/dolfinx/wrappers/geometry.cpp.o
CMakeFiles/cpp.dir/dolfinx/wrappers/graph.cpp.o
CMakeFiles/cpp.dir/dolfinx/wra
ppers/io.cpp.o CMakeFiles/cpp.dir/dolfinx/wrappers/la.cpp.o
CMakeFiles/cpp.dir/dolfinx/wrappers/log.cpp.o
CMakeFiles/cpp.dir/dolfinx/wrappers/mesh.cpp.o
CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o CMakeFiles/c
pp.dir/dolfinx/wrappers/refinement.cpp.o
-Wl,-rpath,/home/dparsons/dolfinx/fenics-dolfinx/debian/tmp-complex/usr/lib/mipsel-linux-gnu:/usr/lib/mipsel-linux-gnu/openmpi/lib:/usr/lib/mipsel-linux-gnu/hdf5/openmpi
:/usr/lib/slepcdir/slepc3.16/mipsel-linux-gnu-complex/lib:/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/lib
/home/dparsons/dolfinx/fenics-dolfinx/debian/tmp-complex/usr/lib/mipsel-linux-gnu/libdolfinx_com
plex.so.0.4.1 /usr/lib/mipsel-linux-gnu/libbasix.so.0.4.2
/usr/lib/mipsel-linux-gnu/libboost_timer.so.1.74.0
/usr/lib/mipsel-linux-gnu/libboost_chrono.so.1.74.0
/usr/lib/mipsel-linux-gnu/openmpi/lib/libmpi_cxx.so
/usr/lib/mipsel-linux-gnu/openmpi/lib/libmpi.so
/usr/lib/mipsel-linux-gnu/hdf5/openmpi/libhdf5.so
/usr/lib/slepcdir/slepc3.16/mipsel-linux-gnu-complex/lib/libslepc_complex.so
/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/lib/libpetsc_complex.so
CMakeFiles/cpp.dir/dolfinx/wrappers/fem.cpp.o: in function
`_PyObject_TypeCheck':
/usr/include/python3.10/object.h:247:(.text+0xa0): relocation truncated
to fit: R_MIPS_CALL16 against `PyType_IsSubtype'
CMakeFiles/cpp.dir/dolfinx/wrappers/fem.cpp.o: in function `_Py_DECREF':
/usr/include/python3.10/object.h:500:(.text+0x174): relocation truncated
to fit: R_MIPS_CALL16 against `_Py_Dealloc'
CMakeFiles/cpp.dir/dolfinx/wrappers/fem.cpp.o: in function
`__Pyx_ImportFunction(_object*, char const*, void (**)(), char const*)':
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:111:(.text+0x46c):
relocation truncated to fit: R_MIPS_CALL16 against
`PyObject_GetAttrString'
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:114:(.text+0x498):
relocation truncated to fit: R_MIPS_CALL16 against
`PyDict_GetItemString'
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:116:(.text+0x4bc):
relocation truncated to fit: R_MIPS_GOT16 against `PyExc_ImportError'
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:116:(.text+0x4c8):
relocation truncated to fit: R_MIPS_CALL16 against `PyModule_GetName'
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:116:(.text+0x4f0):
relocation truncated to fit: R_MIPS_CALL16 against `PyErr_Format'
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:122:(.text+0x514):
relocation truncated to fit: R_MIPS_CALL16 against `PyCapsule_IsValid'
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:123:(.text+0x538):
relocation truncated to fit: R_MIPS_GOT16 against `PyExc_TypeError'
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:123:(.text+0x544):
relocation truncated to fit: R_MIPS_CALL16 against `PyModule_GetName'
/usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:123:(.text+0x560):
additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status
gmake[4]: *** [CMakeFiles/cpp.dir/build.make:266:
../lib.linux-mips-3.10/dolfinx/cpp.cpython-310-mipsel-linux-gnu.so]
Error 1
At this point I'm out of clues. What do you recommend?
Drew
Reply to: