IDE crash when OCR does not find anything. Only 272 unsuccessful calls of text() possible

[Michael Böhm]

Hi!

I am using Sikulix 2.0.4 on Windows 10 64bit, JAVA 11, 2 large sized Monitors (3440x1440 + 2560x1440 as one display, NVIDIA).

Sikulix crashes (meaning: it does not process anymore, does not write logfiles, ALT-SHIFT-C does not work, JAVA needs to be killed in taskmanager to restart sikulix) within following function in about 1 of 10 cases. only when it tries to OCR read text in an area where there is no text:

def text_shrink(Reg): # continously decrease region until OCR gets a match
        texts=False
        Reg=Reg.grow(30,30,0,0)
        while (texts==False or len(texts)<=2) and Reg.getW()>20:
            Reg=Reg.grow(-30,-30,0,0)
            texts=ExtractAlphanumeric_all(Reg.text()).strip() #crash here on OCR
        return texts

The images for this particular run where the crash happend is attached.

Here the debug log (4):

[debug (19.08.20, 10:33:21)] RunTime:loadlib: trying opencv_java342
[debug (19.08.20, 10:33:21)] RunTime:loadLib: opencv_java342.dll already loaded
[debug (19.08.20, 10:33:21)] OCR: start: Tess4J 4.4.1 using Tesseract 4.1.0
[debug (19.08.20, 10:33:21)] RobotDesktop: captureScreen: [824,1291, 224x67]
[debug (19.08.20, 10:33:21)] Image: BufferedImage: (224, 67)
[debug (19.08.20, 10:33:21)] Image: getImage inMemory: __BufferedImage__
[debug (19.08.20, 10:33:21)] Finder2: makeMat: INT_RGB (224x67)
[debug (19.08.20, 10:33:21)] RunTime:loadlib: trying opencv_java342
[debug (19.08.20, 10:33:21)] RunTime:loadLib: opencv_java342.dll already loaded
[debug (19.08.20, 10:33:21)] OCR: start: Tess4J 4.4.1 using Tesseract 4.1.0
[debug (19.08.20, 10:33:21)] RobotDesktop: captureScreen: [854,1291, 164x67]
[debug (19.08.20, 10:33:21)] Image: BufferedImage: (164, 67)
[debug (19.08.20, 10:33:21)] Image: getImage inMemory: __BufferedImage__
[debug (19.08.20, 10:33:21)] Finder2: makeMat: INT_RGB (164x67)
[debug (19.08.20, 10:33:21)] RunTime:loadlib: trying opencv_java342
[debug (19.08.20, 10:33:21)] RunTime:loadLib: opencv_java342.dll already loaded
[debug (19.08.20, 10:33:21)] OCR: start: Tess4J 4.4.1 using Tesseract 4.1.0
[debug (19.08.20, 10:33:21)] RobotDesktop: captureScreen: [884,1291, 104x67]
[debug (19.08.20, 10:33:21)] Image: BufferedImage: (104, 67)
[debug (19.08.20, 10:33:21)] Image: getImage inMemory: __BufferedImage__
[debug (19.08.20, 10:33:21)] Finder2: makeMat: INT_RGB (104x67)
[debug (19.08.20, 10:41:22)] IDE: AbortKey was pressed: aborting all running scripts
[debug (19.08.20, 10:41:31)] IDE: AbortKey was pressed: aborting all running scripts
[debug (19.08.20, 10:43:19)] IDE: AbortKey was pressed: aborting all running scripts
[debug (19.08.20, 10:43:20)] IDE: AbortKey was pressed: aborting all running scripts

Thanks a lot
Michael

[Michael Böhm (badboisikulix)]

Hi!

I can reproduce with following script:

Reg=Region(884,1291,104,64)
Reg.highlight(2)
for i in range(1,1000):
    print_l(i,":",Reg.text())

on my PC it crashed always(!!!) on counter: 272

Here the log:
[debug (19.08.20, 11:19:17)] Finder2: makeMat: INT_RGB (104x64)
[debug (19.08.20, 11:19:17)] RunTime:loadlib: trying opencv_java342
[debug (19.08.20, 11:19:17)] RunTime:loadLib: opencv_java342.dll already loaded
[debug (19.08.20, 11:19:17)] OCR: start: Tess4J 4.4.1 using Tesseract 4.1.0
[debug (19.08.20, 11:19:17)] RobotDesktop: captureScreen: [884,1291, 104x64]
[debug (19.08.20, 11:19:17)] Image: BufferedImage: (104, 64)
[debug (19.08.20, 11:19:17)] Image: getImage inMemory: __BufferedImage__
[debug (19.08.20, 11:19:17)] Finder2: makeMat: INT_RGB (104x64)
[debug (19.08.20, 11:19:18)] RunTime:loadlib: trying opencv_java342
[debug (19.08.20, 11:19:18)] RunTime:loadLib: opencv_java342.dll already loaded
[debug (19.08.20, 11:19:18)] OCR: start: Tess4J 4.4.1 using Tesseract 4.1.0
[debug (19.08.20, 11:19:18)] RobotDesktop: captureScreen: [884,1291, 104x64]
[debug (19.08.20, 11:19:18)] Image: BufferedImage: (104, 64)
[debug (19.08.20, 11:19:18)] Image: getImage inMemory: __BufferedImage__
[debug (19.08.20, 11:19:18)] Finder2: makeMat: INT_RGB (104x64)
[debug (19.08.20, 11:19:47)] IDE: AbortKey was pressed: aborting all running scripts
[debug (19.08.20, 11:19:50)] IDE: AbortKey was pressed: aborting all running scripts

[Michael Böhm (badboisikulix)]

Hi! Some more findings:

Using the "reproduce script":
*) It does NOT crash, when OCR finds a text.
*) It crashes also at counter 272 for different Regions/images, however:
      -) it does not crash (also not later) when I select a Region with "easy" background (not many patterns)
      -) taking a much bigger region results in no difference
      -) finding a word in the meantime makes no difference, the script always crashes at 272
      -) OCR.reset() in the meantime makes no difference, still crash at 272
      -) replace Reg.text() to OCR.readText() - same, still crash at 272
      -) OCR.readLine(), readWord, readChar are also crashing (different count (=128)), even tough they
find characters in any pattern.
      -) Doing OCR alternatively on word and on non-word patterns let it crash earlier (at 172)
      -) Doing OCR on words only does not crash within 1000 runs
      -) OEM=0, PSM=8 also crash
      -) OEM=0, PSM=2 also crash

I hope I found some useful information.

I was not able to find a workaround (other than avoiding OCR on non text).

Good luck
Best Regards
Michael

[RaiMan (raimund-hocke)]

should first be a question