The serialization of the GMF needs to be optional

Bug #838990 reported by Vitor Silva
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenQuake (deprecated)
Fix Released
Critical
beatpanic

Bug Description

The probabilistic event-based calculator is taking about 2 hours to run very simple calculations. It might be due to the fact that it serializes thousands of ground motion fields, in which one is stored per xml file. Even if this task is not delaying significantly the performance of the engine, users should have the option to turn off this serialization. The output folder gets flooded with files which messes up MacFusion or any ftp client.

John Tarter (toh2)
Changed in openquake:
importance: Undecided → Critical
milestone: none → 0.4.4
Revision history for this message
Mattia Barbon (mattia.barbon) wrote :

Tested with the probabilistic_event_based_risk demo (produces ~14K GMFs with 35 points each)

With DB + XML serialization:

real 11m55.672s
user 4m31.949s
sys 0m6.372s

Without DB serialization:

real 10m58.356s
user 3m44.950s
sys 0m5.484s

No serialization (risk reads GMFs from KVS):

real 4m17.530s
user 0m10.329s
sys 0m0.348s

Revision history for this message
Mattia Barbon (mattia.barbon) wrote :

In the comment above, the second entry should read: "With DB serialization:"

Changed in openquake:
status: New → Confirmed
John Tarter (toh2)
Changed in openquake:
status: Confirmed → In Progress
assignee: nobody → beatpanic (kpanic)
Revision history for this message
beatpanic (kpanic) wrote :

https://github.com/gem/openquake/pull/504 modifies the behaviour by adding a GMF_OUTPUT config parameter

beatpanic (kpanic)
Changed in openquake:
status: In Progress → Fix Committed
Revision history for this message
Paul Henshaw (paul-sl-henshaw) wrote :
Changed in openquake:
status: Fix Committed → Won't Fix
status: Won't Fix → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related blueprints

Remote bug watches

Bug watches keep track of this bug in other bug trackers.