X-1.0rc3-931: Java heap space exhausted with long running script using Region.right() .below() .nearby().

Bug #1013396 reported by ente on 2012-06-14
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Sikuli
Critical
RaiMan

Bug Description

******** details - workaround

only Region.right() and .below() and .nearby() are affected.
.above() and .left() are ok.

There is no other workaround, than to make your own functions, to get a Region right or below or nearby of a given region.

-------------------------------------------------------------------------------------------------------------------

It took me a while* to track this one down:

NO memory leak:

def getNumbers(numType, region):
     return 0

def test()
    x=regions['topLeft'].find("JI.png")
    getNumbers("topLeft", x) # <----<----<---- Region

while True:
     test()

MEMORY LEAK:

def getNumbers(numType, region):
     return 0

def test()
    x=regions['topLeft'].find("JI.png")
    getNumbers("topLeft", x.right(75)) #<---<---<--- Region.right(x)

while True:
     test()

System: win7 64bit
JRE 1.6.32 (JRE1.7.04 tested as well)
Sikuli X1.0rc3(905) (1.0RC930 tested as well)

I've tested a few variations of the code above already. I will go on, since that script is essential to a project I am running.

PS: getNumbers is my own text (number) recognition algorithm since region.text() isn't working yet.

* "a while" == 12 days - anyway, sikuli is really good for it's purpose

RaiMan (raimund-hocke) wrote :

--- MEMORY LEAK
How did you detect this?
What are the observed effects?

Well, I have a quite long script which is supposed to run at least 12 hours before restarting. Obviously I had a memory leak: the script stops after 2-4 hours with a jvm heap exception. I've tracked this one down by minimizing my script until nothing had been left. The number recognition is a major part in my script. it is executed about 20 times / minute.

The excerpt provided above starts at about 150 MB private memory and will go up within 5-10 minutes to 250 ... It will end with a heap exception if just let it go on.

RaiMan (raimund-hocke) wrote :

Ok, but this is not a memory leak caused by a specific Sikuli feature, this is some strange behavior of the Java GC (garbage collection), while running a Sikuli script.

The effect of increase of minimum memory consumption is more substantial, when running a script in the IDE compared to running it from command line.

Running a script in IDE directly after IDE start, the base memory consumption jumps from about 120 MB to about 250 MB after having initialized the Sikuli basics and started running the script.

Then memory consumption rises to a value of about base+350MB, before falling back to base (because of GC). The problem is, that base is constantly increasing with each fallback (IDE: +10-30MB, command line using sikuli-script.jar: +0-5MB varying).

So in my opinion, this is a general IDE-script-run problem.

See what happens if you run your script from command line using:
java -jar path-to-sikuli-script.jar path-to-script.sikuli

summary: - Memory leak in Region.right(x)
+ Java heap space exhausted with long running script

Well, I don't agree fully yet. I have prepared two skl-versions of the following script:

def getNumbers(numType, region):
    return 0

def test():
    x=region['3'].find("JI.png")
# y=x.right(75) # <<<<<<<< line activated in "region.right - ML.skl
    y=x # <<<<<<<< line activated in "region.skl"
    getNumbers("topLeft", y)

exactRegion=find("JI.png")
region={}
for i in range(10):
    region[str(i)]=Region(exactRegion.x-100, exactRegion.y-100, 200, 200)
    region[str(i)].setAutoWaitTimeout(0.0)
    region[str(i)].setFindFailedResponse(SKIP)
while True:
    test()

Both scripts are attached. I attached a screenshot as well. Open the screenshot anywhere on your screen using any software (I tested using windows photo viewer). Run both scripts at about the same time and observe their memory usage in windows task manager.

While writing this text, one script went from 145-150 up to 210. That's the ML-script. The other one stays constant at 145-150.

I've figured out the setAutoWaitTimeout(0.0) is important. I guess it's just boosting the number of find-operations per second and thus creating a much easier observable result.

RaiMan (raimund-hocke) wrote :

OK, you are the champ ;-)

The problem is caused behind the scenes by the JPanel preparation for the extra option, to show a popup with the image and the skip/retry/fail option in case of FindFailed.

In the case of Region.right(), every call seems to produce a bunch of new JPanels that are never released and so constantly (slowly though) increase the memory usage.

I used the tool "Java Visual VM" (bundled with the JDK 6), to look at the details.

I tried some variants, but there is no other workaround, than to make your own functions, to get a Region right (or above, below, left, nearby) of a given region.

BTW: neither setAutoWaitTimeout(0.0) nor setFindFailedResponse(SKIP) have any influence on the behavior.

summary: - Java heap space exhausted with long running script
+ Java heap space exhausted with long running script using Region.right()
+ or other spatial operators.

Correction:

only Region.right() and .below() and .nearby() are affected.
.above() and .left() are ok.

summary: Java heap space exhausted with long running script using Region.right()
- or other spatial operators.
+ .below() .nearby().
RaiMan (raimund-hocke) on 2012-06-15
description: updated
Changed in sikuli:
status: New → Confirmed
summary: - Java heap space exhausted with long running script using Region.right()
- .below() .nearby().
+ X-1.0rc3-931: Java heap space exhausted with long running script using
+ Region.right() .below() .nearby().
RaiMan (raimund-hocke) wrote :

I looked into the sources:

It is not the FindFailed option dialog.

The problem is the ScreenHighLighter (the thing that makes the red frames around the screen when capturing or when using Region.highlight.

The 3 affected methods use the "new Region" in a different way, than the other 2 methods. This is a known inconsistency on the Java level of "new Region" (which is tagged as deprecated) and Region.create(), which makes problems when trying to use the observe feature in Java.

The basic problem is, that with Region.right() a new Screen object is created, though already one exists and hence a new ScreenHighlighter object. On the first glance I cannot see, why these Screen and ScreenHiighlighter objects are not released together with the Region objects.

Coming back to my defect with some new observations. I am not a java expert, I am rather a newbie on java.

Some background information on my script: it is doing up to 200+ findAll()-Operations per minute. It consist of a main loop "while 1=1" nested with a few more loops. There is almost no sleep() or wait() operation. Mouse movement is max'ed. AutoWaitTimeout is set to 0.
The observations have been made on the test scripts attached to this defect as well.

It looks like java garbage collection sucks with default options. Adding "import gc" and a few "gc.collect()" within the loops slows down heap exhaust dramatically. For the test scripts it even avoids heap exhaust for more than one hour. My main script is now running stable for at least 12 hours.
I've developed an automatic restart function, i.e. after 12 hours the script leaves it's main loop and starts a windows batch file. The batch file needs a delay at the beginning (using "timeout" or ping) for a few seconds before starting the script again. Otherwise the new instance of my script is executed before java exits. If the new script instance is started before java exists, the memory is not freed but the new instance takes over the old.

John F Leach (jfleach) wrote :

Can one of the Sikuli/Java experts revisit this issue?

We have a simple script which is similar to the code below:

while True:
    while not exists(image_name):
        sleep(0.25)
    click(image_name)

Is there anything we can do to prevent the out of memory issue? If I add gc.collect() within the infinite loop, is it going to fix the problem? This is a major problem when anyone considers using Sikuli for stress and performance testing; adding a wrapper to re-launch Sikuli isn't acceptable.

RaiMan (raimund-hocke) wrote :

@ John
Does exactly the above script produce problems?

If yes, with what heapsize is it running how long? How many iterations?

BTW: this would make it at least live longer, since one find is saved:

while True:
    while not exists(image_name):
        sleep(0.25)
    click(getLastMatch())

RaiMan (raimund-hocke) on 2012-11-02
Changed in sikuli:
status: Confirmed → In Progress
assignee: nobody → RaiMan (raimund-hocke)
milestone: none → x1.0
status: In Progress → Fix Committed
RaiMan (raimund-hocke) on 2012-11-02
tags: added: fkt-nearby
RaiMan (raimund-hocke) on 2012-11-02
tags: added: fkt-region
removed: fkt-nearby
RaiMan (raimund-hocke) on 2013-02-21
Changed in sikuli:
importance: Undecided → High
RaiMan (raimund-hocke) on 2013-05-14
Changed in sikuli:
status: Fix Committed → In Progress
milestone: x1.0 → x1.1
RaiMan (raimund-hocke) on 2014-01-12
Changed in sikuli:
status: In Progress → Fix Committed
RaiMan (raimund-hocke) on 2014-09-17
Changed in sikuli:
importance: High → Critical
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers