smart optimisation in the assistant is non-optimal

[Bruno Postle]

The stages of the existing smart optimisation that happen when you use Align... in the Assistant tab are:

First it optimises just positions, if the final panorama is 360° then it will optimise field of view of the photos in the next step.

It looks at the spread of control points and either optimises just 'b' or 'a,b,c' lens parameters depending on this spread.

If the photos are wider than 60° then d,e will be optimised too.

It does some checks with the result of this second step, if the v,a,b,c,d,e parameters are not credible then it backs out and optimises the project again but with less parameters.

(from SmartOptimise::smartOptimize in src/hugin_base/algorithms/optimizer/PTOptimizer.cpp)

Looking at this code, some of the default thresholds and heuristics could be better:

I think it is safe to optimise field of view if the panorama is greater than about 150° (and if the panorama is not a single stack).

The a,b,c thresholds are much too high.

High a,b,c values can be an indication that the lens type is incorrect, perhaps at this point it would be useful to run another optimisation with fisheye/rectilinear and see which gives the best results.

The straighten function is run but this doesn't work unless there is some horizontal spread of photos (i.e. a vertical panorama fails) see Bug #679282 (sf-2851956)

a,b,c,d,e should never be optimised when the initial alignment indicates that we have a single stack (i.e. the assistant is currently broken with all single stack projects).

The d,e threshold should be a proportion of the photo width rather than a pixel value.

Photometric optimisation is always performed, but this is almost always a mess unless there is a good geometrical alignment. So if the alignment is bad, then photometric optimisation should be skipped.

Photometric alignment needs significant vignetting and/or bracketed exposures to be able to calculate the camera response curve, 'bad' response curves result in 'banding' or 'posterisation' artefacts. The Assistant tab could be refined to back-out of photometric alignment if the vignetting turns out to be low or there is no EV variation.

[rew (r-e-wolff)]

The idea behind "if hfov (pano) > 360 then optimize v(images)" is that if the v(images) is say 10% larger, you'll get a reasonable pano with a 10% larger hfov. i.e. it will grow from say 150 degrees to 165. This does not significantly impact the panorama.

When you WOULD optimize "v", the panorama will "fit" with all possible V values and the optimization run will optimize it to an extreme value like "0".

On the other hand, when the pano is a full 360, a slight deviation in V for the images will cause the panorama not to "fit" around the full circle.

So I don't agree with the suggestion that V can be optimized if the FOV of the pano is larger than 150. It needs to be around 360 degrees to make sense.

[Bruno Postle (brunopostle)]

Regarding optimisation of FoV, the problem with 'v' reducing to zero only happens with very narrow panoramas .

Think about the panorama as segment of the 'sphere'. With a narrow field of view this segment is very similar to a flat surface, and the photos will fit whatever 'v' you give them.

With a scene with a large field of view (>90) the segment is significantly curved and it isn't possible to get a 'better' fit by simply reducing 'v' for the photos.

[tmodes (tmodes)]

Implemented in changesets 9214f2ddb58b and 09790aafd95d.
For the geometric optimizer the limits are set now narrower.
The photometric optimizer does bake up results if the the vignetting is too high and/or the WB values are too high (maybe the limits needs some more fine tuning). It is also possible to add a check for the response curve. But what values for the response curve are considered invalid?