Global Health Observatory

Bug #484779
Comment #7

Comment 7 for bug 484779

Revision history for this message

Jiri Dvorak (jiri-dvorak) wrote on 2009-11-24:

Based on the messages earlier in this thread, the system (especially the Java JVM) is running out of memory. The bad news is that there is not much we can do about it on the application side - barring any unexpected internal bugs or memory leaks, the OH/GHO code memory usage is quite modest, and very little can be gained by additional tuning.

By far, the biggest memory consumer is the Mondrian software component from Pentaho, which is being used by GHO as "black box" (by including mondrian.jar in the build).

Other than adding physical memory to the machine, configuring more "generous" virtual memory in Linux, and maximizing memory availability to the JVM executable (via "java ... -ms ... -mx ..." etc.), the only method of tuning Mondrian resource usage is via the file /who_gho/resources/mondrian.properties . That tuning involved a bit of "black magic" and some testing, and the only official documentation we were able to find is on the URL http://mondrian.pentaho.org/documentation/configuration.php .

Specifically, the following parameters may have immediate influence on Mondrian memory usage (with "sample" values):

- mondrian.query.limit = 4 ... maximum number of simultaneous queries the system will allow

- mondrian.result.limit = 50000 ... maximum size of a result set

- mondrian.rolap.evaluate.MaxEvalDepth = 10 ... maximum number of passes allowable while evaluating an MDX expression

- mondrian.rolap.queryTimeout = 10 ... limits the number of seconds a query executes before it is aborted

- mondrian.rolap.IterationLimit = 10 ... maximum number of iterations allowed when evaluating an aggregate

- mondrian.olap.fun.crossjoin.optimizer.size = ??? ... see the URL, this is pure black magic

- mondrian.rolap.star.disableCaching = true ... setting this to true will clear aggregate cache after each query; that may save a lot of memory

- mondrian.expCache.enable = false ... see the URL, may save some memory, but some MDX queries will run more slowly

- mondrian.rolap.EnableRolapCubeMemberCache ... see the URL, this one probably should stay as "true", but may be worth 1-2 tests ...

- mondrian.util.memoryMonitor.enable = true and mondrian.util.memoryMonitor.percentage.threshold = 60(or 70,80,90) can be used to see memory usage messages earlier in the log file during testing

ALSO PLEASE READ CAREFULLY THE WHOLE SECTION "MEMORY MANAGEMENT" at the URL http://mondrian.pentaho.org/documentation/configuration.php !!!!

Based on the messages earlier in this thread, the system (especially the Java JVM) is running out of memory.  The bad news is that there is not much we can do about it on the application side - barring any unexpected internal bugs or memory leaks, the OH/GHO code memory usage is quite modest, and very little can be gained by additional tuning.

By far, the biggest memory consumer is the Mondrian software component from Pentaho, which is being used by GHO as "black box" (by including mondrian.jar in the build).

Other than adding physical memory to the machine, configuring more "generous" virtual memory in Linux, and maximizing memory availability to the JVM executable (via "java ... -ms ... -mx ..." etc.), the only method of tuning Mondrian resource usage is via the file /who_gho/resources/mondrian.properties .  That tuning involved a bit of "black magic" and some testing, and the only official documentation we were able to find is on the URL http://mondrian.pentaho.org/documentation/configuration.php .