Re: OOM while building ghc 9.4.5
Hi Ilias,
On Wed, Jul 26, 2023 at 12:33 PM Ilias Tsitsimpis <iliastsi@debian.org> wrote:
>
> Hi team,
>
> While preparing ghc 9.4.5 in Debian experimental, it fails to build with
> OOM on mipsel [1].
>
> The command that fails is:
>
> $ "inplace/bin/ghc-stage1" -hisuf p_hi -osuf p_o -hcsuf p_hc -static -prof -H32m -O -lffi -optl-pthread -optc--param -optcggc-min-expand=5 -optc--param -optcggc-min-heapsize=4096 -Wall -this-unit-id ghc-9.4.5 -hide-all-packages -package-env - -i -icompiler/. -icompiler/stage2/build -Icompiler/stage2/build -icompiler/stage2/build/./autogen -Icompiler/stage2/build/./autogen -Icompiler/. -Icompiler/stage2/build/. -optP-DHAVE_INTERNAL_INTERPRETER -optP-DCAN_LOAD_DLL -optP-include -optPcompiler/stage2/build/./autogen/cabal_macros.h -package-id array-0.5.4.0 -package-id base-4.17.1.0 -package-id binary-0.8.9.1 -package-id bytestring-0.11.4.0 -package-id containers-0.6.7 -package-id deepseq-1.4.8.0 -package-id directory-1.3.7.1 -package-id exceptions-0.10.5 -package-id filepath-1.4.2.2 -package-id ghc-boot-9.4.5 -package-id ghc-heap-9.4.5 -package-id ghci-9.4.5 -package-id hpc-0.6.1.0 -package-id process-1.6.16.0 -package-id stm-2.5.1.0 -package-id template-haskell-2.19.0.0 -package-id terminfo-0.4.1.5 -package-id time-1.12.2 -package-id transformers-0.5.6.2 -package-id unix-2.7.3 -Wall -Wno-name-shadowing -Wnoncanonical-monad-instances -Wnoncanonical-monoid-instances -this-unit-id ghc -XHaskell2010 -XNoImplicitPrelude -XBangPatterns -XScopedTypeVariables -XMonoLocalBinds -XTypeOperators -Rghc-timing -Wcpp-undef -Wincomplete-uni-patterns -Wincomplete-record-updates -no-user-package-db -rtsopts -O0 -Wnoncanonical-monad-instances -outputdir compiler/stage2/build -c compiler/./GHC/Hs/Instances.hs -o compiler/stage2/build/GHC/Hs/Instances.p_o -dyno compiler/stage2/build/GHC/Hs/Instances.dyn_o
>
> virtual memory exhausted: Cannot allocate memory
> `gcc' failed in phase `C Compiler'. (Exit code: 1)
>
> Building an unregisterised ghc compiler on 32-bit targets has trouble
> handling the Instances.hs file. This has been reported upstream here[2].
> Until now, we have managed to workaround this by compiling this file
> alone with O0 (as shown in the above command), but since version 9.4.5
> even that doesn't work.
>
> Is there anything else we can try to compile ghc 9.4.5 in mipsel, or
> should we give up in supporting mipsel at all?
>
> [1] https://buildd.debian.org/status/package.php?p=ghc&suite=experimental
> [2] https://gitlab.haskell.org/ghc/ghc/-/issues/17048
Speaking as someone who helps maintain C/C++ libraries; not someone
who packages for Debian or administers the mipsel gear...
I used to run into this alot with the Crypto++ library on ARM gadgets.
"Gadgets" are what I call the dev boards and IoT boards. The dev
boards were low resource, and often had 512MB or 1GB of RAM with no
swap file because of a SDcard. One particular file used to cause an
OOM kill more frequently than I care to remember.
I found two things helped or fixed the problem. First, I would build
with only one make job. That is, 'make -j 1'. 'make -j 3' or 'make -j
5' was sure to break the build.
Second, I would add a swap partition that provided enough space to run
the compiler. For a dev board with 512MB of RAM, I would setup a 2GB
swap partition. I would also set swappiness to 1 or 2 to keep things
in memory. zram would not help because even compressed, the compiler
needed a fair amount of space.
And I do burn through a lot of SDcards. But a new SDcard once or twice
a year is a better outcome than a DoS as the OOM Killer keeps killing
my jobs.
Jeff
Reply to: