enblend creates an unexplainable black area.

[rew]

enblending http://prive.bitwizard.nl/t80001.tif
and http://prive.bitwizard.nl/t80003.tif
results in http://prive.bitwizard.nl/t8a.tif

I cannot say I expected that black triangle in the output. My enblend was pulled from hg this morning (no changes) and built without imagecache or openMP. (imagecache causes corruption of the output similar but different to this. )

[rew (r-e-wolff)]

oops. I see that I forgot to include the commandline I used. I had intended to include it. On the other hand. It can't be simpler....

  enblend t80001.tif t80003.tif -o t8a.tif

[Bruno Postle (brunopostle)]

Yes this is a long-standing enblend bug. I workaround it by changing the order of the photos.

[rew (r-e-wolff)]

I increased the resolution of the output and it went away too. (I had decided not to worry about it for now)

[Thomas Rast (trast)]

I seem to hit this bug on about one in three panoramas I shoot. Can I help in any way in getting it resolved?

[rew (r-e-wolff)]

Thomas,
How good a debugger are you?

This is a bug where a good programmer/debugger just has to spend a day or two finding/fixing it.

This bug resides in the image library that allows the image to be moved to disk if it becomes too large (I forgot its name). So reducing the output image size will reduce/eliminate the problem. I think there is a dangling pointer or memory corruption problem somehow....

The strategy to finding it would be as follows:
* compile your own version from source.
* verify it still happens.
* Find the smallest example that causes the problem.
* verify it doesn't happen when you disable that disk swapping of images. (e.g. add -m <largernumber> on the commandline).
* check out valgrind for memory problems. See if it catches something.
* start debugging.....

[Lukas Wirz (l-wirz)]

I've been getting these artifacts for the last couple of years on at least three different machines whenever I compiled enblend myself.

Interestingly I currently have the situation that the version I compile myself (with openmp and without image_cache) features this bug while the debian dist package (with image cache) doesn't on the same machine. Not sure what we can learn from that fact.

@rew, is that image lib you are talking about a part of enblend or vigra or simething else? I wouldn't call myself a good debugger but I'm annoyed by this bug to such an extend that I want to throw some time at it.

cheers, lukas

[rew (r-e-wolff)]

I used to have the strong impression that it was the image_cache that caused this... Now you are contradicting that feeling.....

If you manage to change the configuration (yes/no openmp ... yes/no imagecache) and isolate the cases where it happens, that would help locate the problem.

Keep in mind that the size of the stitched images matters. And the limits might differ from configuration to configuration. So if you disable openMP your 500Mpixel stitch might suddenly work, because the limit for the bug to appear moved from 400Mpixel to 600Mpixel in that configuration, and not because openmp actually causes the bug.

So, even in this stage there are debugging challenges.

From there, try to "dump" the current in-memory images to see where things go wrong. I expect an unintialized pointer or buffer overrun.

[Lukas Wirz (l-wirz)]

If I recompile without openmp I get exactly the same output. Image_cache has been disabled in 842, so I can't switch that on. I'll spend some time having fun with valgrind.

[rew (r-e-wolff)]

On Sun, Dec 01, 2013 at 07:31:06PM -0000, Lukas Wirz wrote:
> If I recompile without openmp I get exactly the same output.
> Image_cache has been disabled in 842, so I can't switch that on. I'll
> spend some time having fun with valgrind.

Good idea. Valgrind will be able to find problems when you "overstep"
a boundary of a pointer. The effects we're seeing are probably because
a pointer ends up pointing into another pointer's domain. You're not
guaranteed to find it, but the buffer zones between allocations are of
course likely to catch at least some of the boundary issues.....

[Christoph Spiel (cspiel)]

Check out rev 6d6c88dcdc63 or later. Switch off optimization
and -- at your discretion GraphCut -- to get reliable results.

[Lukas Wirz (l-wirz)]

For my testcase (coarse, no-opt, graph-cut) changeset 6d6c88dc doesn't change anything. The worst artifacts only occur with graph-cut ... this agrees with what I see in valgrind: There are a number of errors and the backtrace always goes back to 'A_star' in graphcut. I can, however, not find the place where we read beyond borders or fail to initialize values within the borders.

[Christoph Spiel (cspiel)]

@Lukas: (i) GraphCut is absolutely unchanged in 6d6c88dc,
(ii) after its recent re-implementation it is again WIP
according to its author.

Please run with NFT only to see whether this fundamental
step now works any better. BTW, GraphCut uses NFT as an
educated guess.

If you have a reasonably sized set of images, I'd be
interested in looking at the problem myself.

THX for testing!

[Lukas Wirz (l-wirz)]

My minimal example, a Makefile, and the output I get from 'make test_coarse_nopt_gc' at 6d6c88dc. Now that I look at it again I'm not entirely sure any more that this is the same bug ... I get not so much black corners as black borders.

[Christoph Spiel (cspiel)]

Lukas,

    THX for your minimal example! Admittedly, I cannot
reproduce your "foo.tif" with rev 6d6c88dcdc63, i.e.
current tip. I tried any image order, NFT, GC, w/optimizer,
and wo/optimizer. The result _always_ looked ok and
never showed any black areas. Moreover the seam line
(`--visualize') between the image pairs was sane, too.

Maybe the bug you were hunting for such a long time is
gone now. OTOH, I have example images from Bruno which
let the seamline vectorizer go postal. So, there are some
bugs left that can cause black areas. (Usage of plural
to be on the safe side.)

[kaefert (kaefert)]

For bigger panoramas I'm using a self-build enblend (since the enblend build included in ubuntu fails with larger panoramas with out of memory exceptions although memory is still available)

This newer enblend then will often give me black areas like those:
http://kaefert.is-a-geek.org/misc/hugin/enblend-black-areas/Screenshot%20from%202014-01-30%201918_40.jpg

The only workaround that I have at the moment is to cut out the bad parts with gimp, copy the exif tags from enblend's output to the cut version created with gimp and enblend the appropiate nona-output images again with this cut variant.

Is there something I could do to help getting this problem fixed? (besides working my way through the huge enblend codebase myself)

[kaefert (kaefert)]

just for traceability (especially since this bug seems to still be present [I did not check it myself, just inferring from the bug status])

To explain my comments I've used a few links to pictures hosted on my own NAS reachable through a dyn-dns domain, which is no longer available because they canceled their free services. You can still find the images by replacing the domain kaefert.is-a-geek.org by either koega.no-ip.org or nas.koelbl15.wien.funkfeuer.at
So for example for the image in my comment above this one:
http://koega.no-ip.org/misc/hugin/enblend-black-areas/Screenshot%20from%202014-01-30%201918_40.jpg
http://nas.koelbl15.wien.funkfeuer.at/misc/hugin/enblend-black-areas/Screenshot%20from%202014-01-30%201918_40.jpg

[Peter (s-peter-9)]

I am now fairly successfully stitching maps using hugin on Windows 10. One issue I have is that using enblend often generates black areas in the stitched tiff map. When I reported it at https://groups.google.com/forum/#!topic/hugin-ptx/OdXvko4S5F4 I see this is a known issue.

If anyone wants to explore, I have put 2 zip files in a folder on my DropBox (see below) - one was stitched using enblend (resulting in black areas) and one with the hugin inbuilt stitcher. They both contain 25 image files stitched into one tif. I made no changes other than stitch them with a different stitcher.

The other way I have got round the black area problem has been to make a minor change in hugin and re-stitch using enblend. Sometimes this is succesful in that there are no black areas, other times I have to repeat the process.

2 zip files: https://www.dropbox.com/sh/8z0zf64gupchgj8/AAC6Ckwwv3k6m6vy8-f6Ho7Fa?dl=0

[Bruno Postle (brunopostle)]

I'm attaching a set of ten test cases that reproduce this bug. They are all small images so the whole thing runs in five seconds on this machine.

This has been set up so correct output should be ten identical 100% white images, on this machine I get black artefacts in every output.

[Isaac Qiao (isaacq)]

I fixed this "hole" thing by using command
enblend --pre-assemble --fine-mask --primary-seam-generator=nearest-feature-transform -o result.jpg *.tif

"Preassemble" is for avoiding the warning "no overlapping area between inputs"
"Fine mask" is for a better mask detection
"Nearest feature transform" is for using "fine mask", otherwise the default seam generator "graph cut" will cause error for fine mask (the difference between "graph cut" and "Nearest feature transform" is that the first one will just cut the graphs(faster) and the second one will try to avoid features when cutting graphs(slower))

I think you can also use something like
enblend --pre-assemble --coarse-mask=12 -o result.jpg *.tif
instead of "fine mask",
because by default, enblend use coarse-mask=8, which in my case some times cause black holes. And when I tried to increase the coarse mask value, I got the result with no holes (and a less accurate seam between each adjacent images)(I guess the holes are caused by wrong seam calculation. When I ask enblend for a less accurate seam, as there is less calculation needed, the result is less possible to be wrong, so no holes anymore.).

You could change the value after coarse-mask to see the difference if you increase(less accurate )/decrease(more accurate) it by setting --level=1.(by doing that, you could normally see the seams between images.)
enblend --pre-assemble --level=1 --coarse-mask=12 -o result12.jpg *.tif

[Ken Shirriff (kenshirriff)]

I encountered this problem (a couple random gray squares in my stitched image) with Hugin 2016.2.0.be8da0221960.

I tried the workaround in bug #1795269: add --primary-seam-generator=nft to the enblend parameters, and that made the problem go away.

[PeterPall (peterpall)]

If this problem is symptom of a rounding error in floating-point operations? Sometimes small rounding errors add up to a glitch in a game or a pixel. For details on a place where this happens see Wilkinson's polynomial.

[kaefert (kaefert)]

I just installed hugin 2018.0.0-7 with enblend-enfuse 4.2-4 from the default repositories on my Manjaro (Arch based) Laptop, and stiched an wideangle 360° panorama from 32 images. Have 3 large black areas in output.

I'm now trying the last mentioned workaround "--primary-seam-generator=nft" after I reran the stitching with keeeping the intermediary files.

[kaefert (kaefert)]

update: the parameter "--primary-seam-generator=nft" does seem to make the problem more rare, but right now I have a 360° panorama with a targeted output size of 119448x6727 pixels which even though I tried stiching 3 times always gave me black holes.

I'm currently trying the parameters @isaacq suggested: "--fine-mask --primary-seam-generator=nearest-feature-transform" but that slows down enblend a lot. Already running for over 6 hours.

[kaefert (kaefert)]

Okey so the enblend that takes 5 minutes without the "--fine-mask" parameter took 9 hours with it and gave worse results and without.

Now I instead built enblend myself using this https://aur.archlinux.org/packages/enblend-hg/ with the corrected repository from here: http://hg.code.sf.net/p/enblend/code

And with this I now got a result without any black areas.

[kaefert (kaefert)]

okey so it still randomly happens quite often that I get black areas on stitching. One set that produces black areas though will produce the same black areas when retrying. So I now usually work around this issue by stitching in multiple steps, so split the original ~100 pictures in 5 groups â 20 pictures and if one of them still produces black areas split those in two again, repeat and join successful groups until finished.

[Terrance (terrance-hhh)]

2021-10-21 It still happens
enblend 4.2

command line:

```
enblend -l 3 -f 25280x25280 --compression=NONE -o m.tif --pre-assemble -- m????.tif
```

seems `-l 7` is better than `-l 3`

[tmodes (tmodes)]

Fixes with changes in #2053287

[Ken Shirriff (kenshirriff)]

Thanks so much for fixing this bug. What was the underlying problem?

[ln wirz (lnwirz)]

The bug was what we had discussed on the hugin mailing list four years ago: https://groups.google.com/g/hugin-ptx/c/IgglWhsUCEQ/m/6bUu_YQpCAAJ tldr: one may generate a seam line such that pixels are included from an image that are out of bounds wrt that image.

This fix only affects the blending stage. The seam generation and optimisation are off in more subtly ways, and harder to fix.