Imported GMFs cannot be re-exported

[Michele Simionato]

$ openquake --load-gmf gmf_scenario.xml
<some output ID>
$ openquake --eg <ID> /tmp
  ...
  File "/home/ubuntu/oq-nrmllib/openquake/nrmllib/hazard/writers.py", line 324, in gen_gmfs
    for gmf in gmf_set:
  File "/home/ubuntu/oq-engine/openquake/engine/db/models.py", line 1819, in get_gmfs_scenario
    curs, output.gmf.id, imt, sa_period, sa_damping):
  File "/home/ubuntu/oq-engine/openquake/engine/db/models.py", line 1773, in _get_gmf
    yield site_id, [gmv[r] for r in sorted(rupture_ids)]
KeyError: 10

[Michele Simionato (michele-simionato)]

This inconspicuous error is hiding a serious inconsistency in the importer and in the underlying table structure. The fix will:

- fix the ses_collection table; the field lt_model_id must not be NULL
- when importing a gmf_scenario file the whole table structure of the event based calculator must be populated
- fake ruptures must be generated and inserted in the tables ses_rupture and probabilistic_rupture, and the rupture ids
  in the gmf_data table must be the right ones
- the rupture tags as read from the file must enter in the ses_rupture table correctly
- the ScenarioExporter must be replaced by the EventBasedExporter
- the parameter investigationTime is not defined for scenario computations and must not be exported to the files
- the table ses_collection should point to the trt_model table, not to lt_source_model
- then the field trt_model_id in probabilistic_rupture can be safely dropped
- we need a nontrivial migration procedure to reshuffle the data: in particular new ses_collection records must be
  created (one for each TRT) and the old records must be deleted
- the upgrade manager must be enhanced to improve the usability of Python upgrade scripts
- the procedure loading of the epsilons from the database must be fixed to work with several SES at the same time
- some export routines specific to the scenario case can be removed
- an import/export test must be added
- an old import test in a_tests/risk/scenario/case_1 can be removed (it was not covering the problem anyway)

[Michele Simionato (michele-simionato)]

https://github.com/gem/oq-engine/pull/1545