Issue when compiling WHIZARD using multiple processes via -j

[Giacomo De Pietro]

Few months ago I included WHIZARD (3.1.2) in the external packates used by the Belle II collaboration software.

Our building scheme for the external packages is a bit complicated, but let me briefly summarize here what we do:

- we download the tarball from https://whizard.hepforge.org/downloads/?f=whizard-3.1.2.tar.gz ;
- we "untar" it into `$(EXTBUILDDIR)/whizard`;
- we build it via `cd $(EXTBUILDDIR)/whizard && ./configure --prefix=$(EXTDIR)/whizard --enable-pythia8 --with-pythia8=$(EXTDIR) make && make install`

We started using `make -j NPROCESSES` (where NPROCESSES is defined elsewhere), but we found that sometimes the build stopped due to a concurrency error, so we started using simply `make` to avoid such an issue (which was preventing us from building the external packages in our pipeline).

A collaborator investigated the issue and he found that this patch actually allow us to use `make -j NPROCESSES` for compiling WHIZARD without troubles:

diff --git a/circe2/src/Makefile.in b/circe2/src/Makefile.in
index 9e90c92..9f09795 100644
--- a/circe2/src/Makefile.in
+++ b/circe2/src/Makefile.in
@@ -1283,6 +1283,7 @@ circe2_moments.o: $(MODULE_FILES)
 circe2_ls.o: $(MODULE_FILES)
 circe2_generate.o: $(MODULE_FILES)

+bigarray_compat.cmx: bigarray_compat.cmi
 $(CIRCE2_NATIVE): $(CIRCE2_CMX)
 $(CIRCE2_BYTECODE): $(CIRCE2_CMO)

I am not sure we are the only ones facing the issue, but in case this is a real issue, I hope this report and the included patch helps.

[Juergen Reuter (j.r.reuter)]

Dear Giacomo,
thanks a lot for the report. A quick remark (we will follow up on this in the next collaboration meeting): it seems you are doing an in-source build which should work but is not actively maintained by us and highly discouraged. Indeed, the problem with inconsistent time stamps of sources with this bigarray_compat workaround we noticed ourselves in different incarnations, notably in our CI. Note that this bigarray_compat workaround was necessary to support OCaml compiler versions as old as 4.05 and as new as 4.08+ simultaneously. With v3.1.3 of Whizard (which was released in October 2023 followed by 3.1.4 in November 2023) we now demand as minimal version for OCaml 4.08. Hence, this bigarray_compat workaround is no longer necessary and was removed from the distribution. I do believe that also your fix is no longer necessary in that case (except for retrospectively ease parallel building and installation of v3.1.2 and older). As I said, we will discuss internally and come back to the bug report.
Cheers,
    JRR (Juergen)

[Giacomo De Pietro (giacomo-depietro)]

Thanks for the quick answer!

I will also check in our setup if the issue is solved by moving to Whizard 3.1.4 (we already use a fairly recent version of OCaml).