IndexError: index out of bounds when producing hazard maps

[Paul Henshaw]

Running the "california only" or South East Asia calculations provided by Damiano, the calculation runs to 100% but then fails during the post-processing phase with the following error:

[2012-12-05 10:42:13,714 #27 gemcontrol.ethz.ch PROGRESS MainProcess/19933 root] ** > hazard 99% complete
[2012-12-05 10:42:42,330 #27 gemcontrol.ethz.ch PROGRESS MainProcess/19933 root] ** > hazard calculation 100% complete
[2012-12-05 10:42:42,338 #27 gemcontrol.ethz.ch PROGRESS MainProcess/19933 root] ** post_executing (hazard)
[2012-12-05 10:42:42,469 #27 gemcontrol.ethz.ch PROGRESS MainProcess/19933 root] ** post_processing (hazard)
[2012-12-05 10:41:31,993 #27 gemsun02.ethz.ch CRITICAL PoolWorker-9/8219 root] Error occurred in task: index out of bounds
[2012-12-05 10:41:31,994 #27 gemsun02.ethz.ch ERROR PoolWorker-9/8219 root] index out of bounds
Traceback (most recent call last):
  File "/usr/local/openquake/oq-engine/openquake/utils/tasks.py", line 183, in wrapped
    task_func(*args, **kwargs)
  File "/usr/local/openquake/oq-engine/openquake/calculators/hazard/classical/post_processing.py", line 366, in hazard_curves_to_hazard_map
    imls = hazard_maps[i]
IndexError: index out of bounds
[2012-12-05 10:42:42,994 #27 gemcontrol.ethz.ch CRITICAL MainProcess/19933 root] Calculation failed with exception: 'index out of bounds'
Traceback (most recent call last):
  File "./bin/openquake", line 465, in <module>
    main()
  File "./bin/openquake", line 445, in main
    args.force_inputs, args.exports)
  File "./bin/openquake", line 219, in run_hazard
    completed_job = engine2.run_hazard(job, log_level, log_file, exports)
  File "/usr/local/openquake/oq-engine/openquake/engine2.py", line 336, in run_hazard
    return _run_calc(job, log_level, log_file, exports, calc, 'hazard')
  File "/usr/local/openquake/oq-engine/openquake/engine2.py", line 403, in _run_calc
    _do_run_calc(job, exports, calc, job_type)
  File "/usr/local/openquake/oq-engine/openquake/engine2.py", line 485, in _do_run_calc
    calc.post_process()
  File "/usr/local/openquake/oq-engine/openquake/calculators/hazard/classical/core.py", line 355, in post_process
    post_processing.do_hazard_map_post_process(self.job)
  File "/usr/local/openquake/oq-engine/openquake/calculators/hazard/classical/post_processing.py", line 449, in do_hazard_map_post_process
    utils_tasks._check_exception(results)
  File "/usr/local/openquake/oq-engine/openquake/utils/tasks.py", line 98, in _check_exception
    for result in results:
  File "/usr/lib/python2.7/dist-packages/celery/result.py", line 350, in iterate
    propagate=propagate)
  File "/usr/lib/python2.7/dist-packages/celery/result.py", line 95, in get
    interval=interval)
  File "/usr/lib/python2.7/dist-packages/celery/backends/amqp.py", line 147, in wait_for
    raise self.exception_to_python(meta["result"])
IndexError: index out of bounds

[Lars Butler (lars-butler)]

This bug can be reproduced with any classical hazard calculation using the following parameters:

number_of_logic_tree_samples = 1
mean_hazard_curves = false
quantile_hazard_curves =
poes_hazard_maps = at least one PoE

This is issue is caused by an uncommitted transaction in https://github.com/gem/oq-engine/blob/ccdb6713b8a2365ba7414b25fef836453248457b/openquake/calculators/hazard/classical/core.py#L267.

I have submitted this patch to address the issue: https://github.com/gem/oq-engine/pull/971

Note: https://github.com/gem/oq-engine/pull/968 also includes a different fix for the same issue, but 971 address the issue more directly.