hugin invokation of make failure on MAKEFLAGS=-j2

[Andreas Metzler]

Hello,

this has originally been reported by Helmut Grohne in <http://bugs.debian.org/525494>.
----------------------------------
nona -z PACKBITS -r ldr -m TIFF_m -o blah -i 0 /tmp/huginpto_deFg2s
nona -z PACKBITS -r ldr -m TIFF_m -o blah -i 1 /tmp/huginpto_deFg2s
nona -z PACKBITS -r ldr -m TIFF_m -o blah -i 2 /tmp/huginpto_deFg2s
rm blah0000.tif blah0001.tif blah0002.tif
enblend --compression LZW -m 1500 -f2277x4564 -o blah.tif blah0000.tif blah0001.tif blah0002.tif

enblend: error opening input file "blah0000.tif":

Precondition violation!
Unable to open file 'blah0000.tif'.

make: *** [blah.tif] Fehler 1
----------------------------------

Hugin stitiching breaks if MAKEFLAGS=-j2 is set. The reason is a rather simple one, during stitching hugin invokes "make all clean". With parallelisation clean is run before all is finished and this happens with three images and -j2 (xa and xb are run in parallel):
1a nona for image 1
1b nona for image 2
------
2a nona for image 3
2b rm temporary files, including the output the first nona invocations
------
3a run enblend, which fails due to missing files.

The correct fix is to only run clean after all has finished (make all && make clean). Removing MAKEFLAGS from the environment might not hurt but should be configurable.
See also <https://sourceforge.net/tracker/?func=detail&aid=2781240&group_id=77506&atid=550441>.

This is verified for 0.7.0, but afaict this should also apply to SVN HEAD (0.8.0 prerelease r3801), which invokes "make test all clean".

thanks, cu andreas

[Yuv (yuv)]

can anybody verify if this still happens with the new Makefilelib in 2010.4 ?

[Andreas Metzler (k-launchpad-downhill-at-eu-org)]

Verified in 2010.4.0. Hugin nowadays invokes "make -f ... info test all clean", but this does not make a difference, "clean" and "all" still run in parallell.

[Launchpad Janitor (janitor)]

[Expired for Hugin because there has been no activity for 60 days.]

[Andreas Metzler (k-launchpad-downhill-at-eu-org)]

Still present in 2011.0.0 beta3. It invokes "make -f /tmp/huginmk_pViQ5m info test all clean"

[tmodes (tmodes)]

I'm not sure if running several instances of nona or enblend parallel is a good idea. Both program can use several threads/core parallel. So running several instance of nona parallel can result in unwanted blocking behavior of the different threads. Also the programs needs often a lot of free memory. So when running several threads parallel this is amplified. This can be to relieved to some amount by restricting the number of threads in the preferences.
Some idea: Would it help to call make with the -j1 switch, also "make -f makefile -j1 info test all clean"? Or do you think, that's a wrong way?

[Andreas Metzler (k-launchpad-downhill-at-eu-org)]

On 2012-03-06 tmodes <email address hidden> wrote:
> Previously Andreas wrote:
[...]
>> The correct fix is to only run clean after all has finished (make all && make clean). Removing MAKEFLAGS from the environment might not hurt but should be configurable.
>> See also <https://sourceforge.net/tracker/?func=detail&aid=2781240&group_id=77506&atid=550441>.
[...]
> I'm not sure if running several instances of nona or enblend parallel is a good idea. Both program can use several threads/core parallel. So running several instance of nona parallel can result in unwanted blocking behavior of the different threads. Also the programs needs often a lot of free memory. So when running several threads parallel this is amplified. This can be to relieved to some amount by restricting the number of threads in the preferences.
> Some idea: Would it help to call make with the -j1 switch, also "make -f makefile -j1 info test all clean"? Or do you think, that's a wrong way?
[...]

Hello,
Afaict using the -j1 switch would work, at least with my version of
make commandline arguments supercede MAKEFLAGS:
ametzler@argenau:/tmp$ env MAKEFLAGS=j2 make foo bar
foobarfoo done
bar done
ametzler@argenau:/tmp$ env MAKEFLAGS=j2 make -j1 foo bar
foofoo done
barbar done

It seems to be a matter personal preference which of the (combination)
of proposed fixes is the best one:
* pass -j1
* Don't run make in a -j2 incompatible way, use "make all && make
  clean" instead of "make all clean"
* Drop MAKEFLAGS from make's environment.

cu andreas

[tmodes (tmodes)]

> It seems to be a matter personal preference which of the (combination)
> of proposed fixes is the best one:
> * pass -j1

Commited in rev. 76f04d672b03

>* Don't run make in a -j2 incompatible way, use "make all && make
> clean" instead of "make all clean"
This would require some more changes to be compatible with the progress dialog. So I decided for the easier way ;-)