Stereo images alignment

[Vladimir Nadvornik]

Hi,
I tried to use align-image-stack for processing stereo images (taken from slightly different angle). Here are the modifications that I did:

1. Two-phase control point matching - the patch adds second phase, where the points are fine-tuned on full-size image. This gives better results with comparable speed.

2. stereo mode - the points are optimized only for Y (horizontal line). Horizontal shift of the control points is allowed. One vertical line is added to anchor the horizontal shift of the images. The patch also adds options to optimize distortion and image center shift.

3. extra control points around the image edges - in stereo mode the image edges are interesting for "stereo window" alignment. The normal method generated too few control points near the edges. This patch forces additional points, which are then filtered by correlation and geometric pruning.
I want to work on automatic "stereo window" alignment in the future.

[Vladimir Nadvornik (nadvornik-suse)]

The file align_image_stack-stereo.tar.gz was added: stereo patches

[Vladimir Nadvornik (nadvornik-suse)]

Hi,

Here are more related patches:

4. Modification of crop algorithm, that woks on intersection of image area instead of an union. Btw, this is the algorithm requested in bug #679981 if I understand it correctly.

[Vladimir Nadvornik (nadvornik-suse)]

5. Algorithm for stereo window alignment and autocrop, with the corresponding commandline options.

[Vladimir Nadvornik (nadvornik-suse)]

6. The pto file was written without output format options, so it was not directly usable for reproducing the output. This patch fixes it. IMHO an even better fix would be possible at the cost of breaking backward compatibility.

[Vladimir Nadvornik (nadvornik-suse)]

There is still a problem that the pto file with optimized distortion and shift is not correctly read into hugin. Manually creating a new lens for the second image fixes it. I can't tell if the bug is in hugin or in allign-image-stack.

[Vladimir Nadvornik (nadvornik-suse)]

And here is an example of use:

align_image_stack -a "aligned" -i -P -C left.jpg right.jpg
composite -stereo 0 aligned0000.tif aligned0001.tif anaglyph.jpg

[Yuv (yuv)]

On January 13, 2011 02:43:59 am Vladimir Nadvornik wrote:
> And here is an example of use:
>
> align_image_stack -a "aligned" -i -P -C left.jpg right.jpg
> composite -stereo 0 aligned0000.tif aligned0001.tif anaglyph.jpg

Hi Vladimir

thank you for your contribution. do I understand correctly that your patches
add new options to align_image_stack without changing the existing behavior /
options?

I don't have much time to test other than that it does not break the build;
and I do not have a set of stereo images to try anyway. But if you tell me
that the patch does not change existing functionality to existing users, we
can get it into the default codeline where it will get more exposure /
testing.

Yuv

[Vladimir Nadvornik (nadvornik-suse)]

Patch no. 1 changes the matching algorithm. On images that I tested it gave the same or better results, but in theory there can be some regression with some images.
Patch no. 6 changes the written pto files because the original behavior was IMHO buggy.

Otherwise there are no changes for existing users.

[Vladimir Nadvornik (nadvornik-suse)]

Here is more detailed usage description:
http://vndlinuxphoto.blogspot.com/2011/01/stereo-image-alignment-in-hugin.html

[Yuv (yuv)]

Hi Vladimir,

I added your patches to a new branch in the repository so that they can be more widely tested / improved by the community. I will be announcing it on Hugin-PTX shortly.

http://hugin.hg.sourceforge.net/hgweb/hugin/hugin/rev/36e7649cf137

Some feedback:

patch4 works for align_image_stack but not for HDR panoramas in Hugin. If I read your patch correctly, you introduced a boolean variable, 'intersection', to identify if the point must be inside a whole stack of images; or if it merely must be inside a single image. However the test is for the point to be inside all images, and not inside a stack. align_image_stack is a special case in that there is only one stack. The general case of an HDR panorama has multiple stacks; and to fix bug #679981 the test would need to be limited to each single stack separately; as opposed to all images (= all stacks).

Instead of iterating all images with

for(UIntSet::const_iterator it=activeImages.begin();it!=activeImages.end();it++)

you want to iterate all stacks; and within each stack run the inside/outside test.

The above is an approximation, though it may be good enough. It assumes that the stacks are perfectly aligned and might miss on some marginal pixels with hand held input.

To do it perfectly, you would want to iterate all images within an exposure layer and make sure the point is inside each exposure layer, but this has its limitations as well if the camera was on auto-exposure: the result would depend on the definition of the EV range to be considered a single layer. I personally would try first to iterate the stacks.

Last but not least, next time you produce a patch can you simply use `hg diff > patch.file` to generate it? your patch file did not have the full path information for the patched files, requiring more manual work than necessary to apply them.

Thank you again for your contribution
Yuv