Major performance problem when viewing big scenarios in high resolution
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
openWNS Wrowser |
Confirmed
|
Medium
|
Ramin Rezai Rad |
Bug Description
When viewing a scenario in wrowser, the updateFileList() method can take a long time consuming gigabytes of memory if the scenario is big and/or the scan output files have a high resolution. The reason for this is that to display the available probe names, all probe *contents* are also parsed and read into memory.
I propose this patch to fix it:
=== modified file 'wrowser/
--- wrowser/
+++ wrowser/
@@ -102,17 +102,20 @@
import wrowser.Probe
- if os.path.
- self.viewScenar
- else:
- self.viewScenar
+ self.viewScenar
+ dirname = self.workingDir + '/output/'
+ if os.path.
+ for ff in os.listdir(
+ filename = os.path.
+ if os.path.
+ for suffix in wrowser.
+ if suffix in filename:
+ self.viewScenar
- doNotShowThese = []
- for k,v in self.viewScenar
- if v.probeType is not 'Table':
- doNotShowThese.
- for k in doNotShowThese:
- self.viewScenar
+ ### Here we basically only want to read the names of all table probes. The way this was
+ ### written meant that all probes (including all contents) were read into memory only to
+ ### throw away everything except the Table probenames. In the process, Gigabytes of memory
+ ### were used and the parsing took several minutes for bigger scenarios.
Changed in openwns-wrowser: | |
assignee: | nobody → Ramin Rezai Rad (rrr-comnets) |
Thank you for that one. We use a similar approach inside Probes.py when dealing with PDF probes. We do not read the histogram data as long as it is not accessed. Maybe this could be the place to do the changes for TableProbes as done in getHistogram of the PDF probe.
endswith() instead of "in" should be used to compare the suffix.
My suggestion: Call TableProbe. readProbes( dirname) rather than readAllProbes, change TableProbe to not read data unless required.