Enfuse brightness mismatch in deghosted area

[Thomas Baumann]

Enfuse results in a brightness mismatch of dark deghosted areas when exposure-mu is decreased from its default value.

Program Version:
- enfuse 4.1.3 (could not compile 4.2)

Platforms:
- Debian Jessie
- OS X 10.11.6 via MacPorts

== Testcase Files ==

https://www.dropbox.com/sh/8la13xukmfamjk5/AACI6mruatMEsZ27wZgl8ysha?dl=0

Bracketed exposures:
161022_tc2_7079.tif - RGBA
161022_tc2_7081.tif - RGBA
161022_tc2_7083.tif - RGBA
161022_tc2_7085.tif - RGBA
161022_tc2_7086.tif - RGB

Enfused results:
161022_mu00.tif - exposure-mu=0.0
161022_mu02.tif - exposure-mu=0.2
161022_mu05.tif - exposure-mu=0.5 (default)

Commands to reproduce:
$ enfuse --exposure-mu=0.0 -o 161022_mu00.tif 161022_tc2_70??.tif
$ enfuse --exposure-mu=0.2 -o 161022_mu02.tif 161022_tc2_70??.tif
$ enfuse --exposure-mu=0.5 -o 161022_mu05.tif 161022_tc2_70??.tif

== Detailed Description ==

Deghosting;
All exposures except the lightest one (..7086) share the same alpha channel which masks a dark area in the middle of the image for deghosting of moving leaves. The masking is binary, there are no partially masked pixels.

When I change the exposure-mu parameter from its default 0.5 to a smaller value to darken the image, the deghosted area stays much lighter than its surroundings. The problem does not occur when deghosting a bright area using pixels from a dark exposure.

[Christoph Spiel (cspiel)]

Thank you for a report in exemplary form. The small input image sizes
in particular are highly appreciated!

Enfuse weights each *overlapping* pixel of the input images at that very
pixel. Naturally, if there is no overlap, there is nothing to weight.
The extreme case is no pixels at all, a hole in the image and then your
case where only a single image participates because of masking out all
others or, generally no image overlap. Enfuse (and Enblend) simply copy
solitary pixels to the output image *unchanged*. We could even issue a
warning on this case: "Nothing to fuse" or "Nothing to blend".

In your example the problem is homegrown by masking out all but one
image. The luminance match for the default exposure optimum stems from
the non-masked image falling well into the maximum of the Gaussian with
mu=.5. If you want to stick with your workflow and still want a
different mu, choose as un-masked image the one that best matches your
desired mu.

[Thomas Baumann (thomas-t)]

Thank you for your explanation. I am not quite with you, though. Sure, there is nothing to weight and mix in the deghosted area with pixels from only one exposure, but nevertheless brightness and contrast of this area follow nicely when I increase exposure-mu. Perhaps this adaptation mechanism at the border ceases to work when the exposure is too far off. There might even be a built-in limit to prevent side-effects like increased noise.

Contrary to my former statement, there is a symmetrical effect when deghosting a bright area with pixels from the darkest exposure. This can be seen in the new combined testcase tc3:

$ enfuse --saturation-weight=0 --exposure-mu=0.0 -o 161022_tc3_mu00.tif 161022_tc3_70??.tif
$ enfuse --saturation-weight=0 --exposure-mu=0.2 -o 161022_tc3_mu02.tif 161022_tc3_70??.tif
$ enfuse --saturation-weight=0 --exposure-mu=0.5 -o 161022_tc3_mu05.tif 161022_tc3_70??.tif
$ enfuse --saturation-weight=0 --exposure-mu=0.8 -o 161022_tc3_mu08.tif 161022_tc3_70??.tif
$ enfuse --saturation-weight=0 --exposure-mu=1.0 -o 161022_tc3_mu10.tif 161022_tc3_70??.tif

[Christoph Spiel (cspiel)]

> Perhaps this adaptation mechanism at the border ceases
> to work when the exposure is too far off. There might
> even be a built-in limit to prevent side-effects like
> increased noise.

Neither is there an `adaptation mechanism' nor a `built-in limit'.
What you see is just the Mertens/Kautz/van Reeth algorithm at
work. If you dig deeper in this issue database you may even
find a bug report where Enblend did not exactly reproduce the
parts of the images that were not overlapping.

Please review change
    http://hg.code.sf.net/p/enblend/code/rev/04948f29c8ed
where I added some clarifying paragraphs to the documentation.

[Thomas Baumann (thomas-t)]

> What you see is just the Mertens/Kautz/van Reeth algorithm
> at work.

That is amazing, thanks for the clarification. The
documentation should make that clear now.

[Christoph Spiel (cspiel)]

Thanks for reviewing the patch!

Just to demonstrate the power of the multi-level blending:
Again set mu=0 and _reduce_ the number of blend levels with
option `--levels'. You'll find your masked tree sticking
out of the darkness much more prominently. By default
both Enblend and Enfuse maximize the number of levels for
each overlapping region (see documentation for details).

[Christoph Spiel (cspiel)]

I stand corrected. Thanks go to Thomas for poining out
what I ignored!

The documentation was corrected accordingly in
rev f0304648cc0f. See
    http://hg.code.sf.net/p/enblend/code/rev/f0304648cc0f

The crucial change is the restriction in the sentence
"Enblend and Enfuse simply copy the pixels of this
region to the output image ***if they are away far
enough from the overlap area***."

In particular, my explanation for your Enfuse example was
wrong. All your images overlap completely and the small
masked out portion is certainly influenced by all other
images through the Gaussian pyramids as explained in the
fixed documentation.

Sorry for the mess.