Enable sampling of the epsilon matrix

Bug #1338011 reported by Michele Simionato
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenQuake Engine
Fix Released
High
Michele Simionato

Bug Description

The event based risk calculator need to transfer around a huge matrix of floats (the so called epsilon matrix) of shape (num_assets, num_ruptures). The data transfer is killing rabbitmq. At the moment we are dealing with it by enabling the `epsilons_management=fast` flag that produces just one epsilon per asset. Vitor suggested that we do sampling of the epsilons instead, i.e. we let the user decide how many epsilons should be generated per asset. So we should remove the parameter `epsilons_management` and introduce a numeric parameter `epsilon_sampling` instead. When it is zero, the sampling is disabled, i.e. we produce a number of epsilons equal to the number of ruptures. Otherwise we produce a number of epsilons per asset at most equal to `epsilon_sampling` (if there are less ruptures than samplings we take the number of ruptures as minimum).

NB: the epsilon sampling mechanism only work for the event based calculator. For the scenario_risk calculator the number of realizations is usually small (a few thousands, not hundreds of thousands) so that we are typically able to run our computations without sampling. Should the epsilon transfer problem arises there, we will need to add the sampling even in that case.

Revision history for this message
Michele Simionato (michele-simionato) wrote :
Changed in oq-engine:
status: New → In Progress
importance: Undecided → High
assignee: nobody → Michele Simionato (michele-simionato)
milestone: none → 1.0.1
Changed in oq-engine:
status: In Progress → Fix Committed
Changed in oq-engine:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.