Rewrite the classical calculator as a map-reduce job

Bug #1256471 reported by Michele Simionato
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenQuake Engine
Fix Released
High
Michele Simionato

Bug Description

Recently we proved that passing large amounts of data via celery/rabbitmq is as efficient as storing them in the database, if not better. Then the current approach used in the classical calculator, in which partial results are stored in the database, locking the hazard_curve_progress table and thus serializing the computation, has no justification. Hundreds of lines of code can be removed and the calculator can be made both simpler and more efficient by using a standard map-reduce approach in which the arrays are kept in memory and aggregated in the controller node, not in the database.

Bonus point: one can also remove the special case complication of point sources.

Changed in oq-engine:
status: New → In Progress
importance: Undecided → High
assignee: nobody → Michele Simionato (michele-simionato)
milestone: none → 1.0.1
Revision history for this message
Michele Simionato (michele-simionato) wrote :
Changed in oq-engine:
status: In Progress → Fix Committed
Changed in oq-engine:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.