Remove the duplicate entries in hzrdr.ses_rupture
Bug #1266641 reported by
Michele Simionato
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenQuake Engine |
Fix Released
|
High
|
Michele Simionato |
Bug Description
In the event based calculator, the generated ruptures ARE THE SAME FOR ALL SESs, however we store lots of copies of them without reason. The sensible solution would be to store the ruptures only once, and to add a repetition number, to count how many copies are generated: that number is the one returned by the method rupture.
1 SES per 10,000 years.
description: | updated |
Changed in oq-engine: | |
assignee: | nobody → Michele Simionato (michele-simionato) |
importance: | Undecided → High |
milestone: | none → 1.0.1 |
description: | updated |
Changed in oq-engine: | |
status: | In Progress → Fix Committed |
Changed in oq-engine: | |
status: | Fix Committed → Fix Released |
To post a comment you must log in.
Here is an analysis performed on gemsun02, in the branch removing-ruptures-2 (date 2014-02-22), for Miriam's island:
operation tot_duration pymemory counts ses_and_ gmfs 11,716 1,855 57
total compute_
saving ses 5,128 None 57
computing gmfs 3,823 None 57
saving gmfs 2,578 24 57
generating ruptures 105 None 57
filtering ruptures 44 None 57
initialize_sources 12 4.094 1
As you see, "saving ses" is the dominating operation with nearly half of the time spent there (and it is worse in the cluster).
The time spent there is wasted in saving copies. There are 243202 ruptures, but actually only 11091 of them are distinct ruptures, 95.4% of them are copies.