Complete the work to instrument the calculators with the EnginePerformanceMonitor by adding a few missing calculators

Bug #1171060 reported by Michele Simionato
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenQuake Engine
Fix Released
High
Michele Simionato

Bug Description

Unfortunately our knowledge of the performance characteristics of the different calculators is very rudimentary, for lack of realistic use cases. In the table below I have marked with a question mark the cases were I did not run a performance analysis in a realistic situation. If there is no question mark it means I have run a performance analysis in a realistic situation, but in practice this was done for a single situation (i.e. the Kathmandu demo for scenario and scenario_damage, the California simulation for event_based, etc),
so you should not necessarily believe the results. For instance, for event_based hazard, the performance of Miriam's computation (very database intensive) was completely different from the performance of the California example. Having said that, here is the situation:

Hazard calculators

name | dominated by
-----------+-------------
classical | computation
event_based| computation
disagg | computation
scenario | database

Risk calculators

name | dominated by
---------------+-------------
classical | database?
classical_bcr | database?
event_based | database
event_based_bcr| database?
scenario | database
scenario_damage| database

The ticket requires instrumenting all the calculators so that we may confirm such findings in other realistic cases. If it is confirmed that the risk calculators are always dominated by the database (in particular the transfer time for large arrays) perhaps we may consider different strategies: for instance do not distribute the computation and perform everything in the database with stored procedures.

Revision history for this message
Michele Simionato (michele-simionato) wrote :

Done little by little in different pull requests

Changed in oq-engine:
assignee: nobody → Michele Simionato (michele-simionato)
importance: Undecided → High
status: New → Fix Committed
milestone: none → 1.0.0
Changed in oq-engine:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.