Complete the work to instrument the calculators with the EnginePerformanceMonitor by adding a few missing calculators
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenQuake Engine |
Fix Released
|
High
|
Michele Simionato |
Bug Description
Unfortunately our knowledge of the performance characteristics of the different calculators is very rudimentary, for lack of realistic use cases. In the table below I have marked with a question mark the cases were I did not run a performance analysis in a realistic situation. If there is no question mark it means I have run a performance analysis in a realistic situation, but in practice this was done for a single situation (i.e. the Kathmandu demo for scenario and scenario_damage, the California simulation for event_based, etc),
so you should not necessarily believe the results. For instance, for event_based hazard, the performance of Miriam's computation (very database intensive) was completely different from the performance of the California example. Having said that, here is the situation:
Hazard calculators
name | dominated by
-------
classical | computation
event_based| computation
disagg | computation
scenario | database
Risk calculators
name | dominated by
-------
classical | database?
classical_bcr | database?
event_based | database
event_based_bcr| database?
scenario | database
scenario_damage| database
The ticket requires instrumenting all the calculators so that we may confirm such findings in other realistic cases. If it is confirmed that the risk calculators are always dominated by the database (in particular the transfer time for large arrays) perhaps we may consider different strategies: for instance do not distribute the computation and perform everything in the database with stored procedures.
Changed in oq-engine: | |
status: | Fix Committed → Fix Released |
Done little by little in different pull requests