Some more details. The code using parmap is this one:
let instsets = if num_cores > 1 then
(* On 32 bit architectures it can happen that the result returned
* by parmap is too big to fit into the OCaml constraints for
* maximum allocatable memory. Should this problem occur, we
* repeat the computation with smaller chunk sizes.
* Unfortunately, should the result be too big and the child
* process throws an Out_of_memory error, then all the parent
* process sees is the Failure exception that the marshalling
* module throws. Thus, the Failure we do catch might also be
* caused by other problems than an Out_of_memory exception.
* https://github.com/rdicosmo/parmap/issues/46
*)
let rec splitjob maxsplits todo =
try
Parmap.parmap ~ncores:num_cores worker (Parmap.L todo)
with
| Failure "input_value_from_block: bad object" -> begin
if maxsplits <= 0 then fatal "exceeded maximum split depth";
warning "parmap raised out of memory exception with list length %d, splitting job..." (List.length todo);
let l1, l2 = List.split_nth ((List.length todo)/2) todo in
let res1 = splitjob (maxsplits-1) l1 in
let res2 = splitjob (maxsplits-1) l2 in
List.append res1 res2
end
in
let maxsplits = 4 in
splitjob maxsplits todo
else
List.map worker todo
in
I already had problems with parmap in the past which is why I added this
workaround for when large lists yielded an exception on 32 bit architectures.
Now, after the ocaml update, a Out_of_memory exception is thrown instead of the
Failure above but even when catching and handling that, a segfault still
happens. The segfault also happens when creating *.d.byte files.
So somewhere a behavioural change happened between 4.05.0 and 4.08.1 which
affects parmap in this way on 32 bit architectures.
Thanks!
cheers, josch
Attachment:
signature.asc
Description: signature