findAll limited to 100 matches

[Raffles]

Calls to findAll seem to report at most 100 matches., silently discarding any further matches. Suggest findAll be configurable to specify the maximum number of matches, even if the default is 100, in the same way that similarity defaults to 0.7 but can be configured. Also suggest that if the max is exceeded, an exception is thrown so that missing matches can be spotted easily

[Raffles (robert-finking)]

There was a question about this here: https://answers.launchpad.net/sikuli/+question/172529

Unfortunately the answer given no longer works:

   Best Tsung-Hsiang Chang (vgod) said on 2011-09-28: #1

   This's not documented yet, but there's a parameter "FindAllMaxReturn" to change this.
   For example,
   from org.sikuli.script.natives import Vision
   Vision.setParameter("FindAllMaxReturn", 1000)

[RaiMan (raimund-hocke)]

Do you have an example, where findAll() returning 100 matches or even more might be of value?

[Raffles (robert-finking)]

A clickable grid.

[RaiMan (raimund-hocke)]

a grid is a grid, because it has some regular row/column structure.
If you act on a grid with Sikuli, you somehow have to evaluate this structure, if it is not known anyway or might vary.

the row and column structure can be evaluated best with some find/findAll in horizontal/vertical direction in limited space (left column/ top row).
If the grid has even elements, this is sufficient. If it does not have even elements, findAll would not help anyway.

Since in a grid you usually want to access a cell, this is best represented by a pair of row and column number like
click(myGrid(row, column))

... and I think, the evaluation to be done in the function myGrid is much more complicated with the results of a findAll than with the above approach.

BTW: version 1.1.0 has a feature to define a grid on a region and then access rows, columns and cells as subregions.

... and there are many people out there, that invest in complicated scripting in clicking around in Excel sheets, because they do not know, that there are powerful libraries to access the sheet content directly (like xlrd for Python/Jython)

conclusion: I do not agree, that a grid is a suitable example for the need of a findAll returning hundreds of matches.

[skeets (skeets-malloy)]

This also affects me too in a similar fashion.

I have an image of a certain size and it is a blank image (16x25, RGB is 0, 0, 0). I find a larger (parent) image using Region exists method (244x25). Then from that parent image, I do a sub-verify using the findAll method to find the smaller blank image. My blank image can be found in many places within the parent image. The location within the parent image that it should find it at is 0,0 but it never finds it at that location. The closest it seems to find it is at 2, 0 (which is also a valid location to find the blank 16x25 image). I had a larger parent image that I scaled down because I too thought that the number of finds is limited to some number. The scaled down version STILL does not find the child image within the parent image at 0,0... it only finds it at other locations... which is now limited to maybe 35 places... If I change up my images and use any child image other than a blank image it seems to find the image at the location I expect.

[RaiMan (raimund-hocke)]

@skeets
Yep, that problem is builtin with findAll(): in some margin around a match another match will not be looked for (speed), which makes sense for non-solid images. If an image can be found more than once in a region, it is not predictable (due to the underlying OpenCV feature matchTemplate()) in what sequence they will be found.
For solid images these 2 effects together lead to your experience.
Even when looking for a solid image in a larger solid region, the match location is not predictable and might not be reproducible.
There will be no change for that behaviour in version 1.

I am not sure wether it changes the situation, but did you try with Pattern(image).exact() instead of just image?

[skeets (skeets-malloy)]

OK thanks for the info and suggestion!! Makes more sense now.

For us, we need to be able to find the expected (child) image at the expected coordinates within the parent image for our test to pass. Those expected coordinates may not always be at 0,0 within the parent image. So in order to find the child image (the blank image in question), we thought we needed to use the findall method since in this case the child image can be found in many places within our parent image. So we loop through all of the findall matches testing the match coordinates against our expected coordinates. So it doesn't seem to me that we could use your suggestion if I understand correctly? Unfortunately we can't scale down our parent image any further or else it will not be unique enough on our screen. BUT we can update our "blank" image to not be blank anymore. We are testing a field to see if it has an icon or not and that field has a neighbor field that is never blank so think we can also get around this issue with a bigger child image that includes part of the neighbor field.

[RaiMan (raimund-hocke)]

@skeets
Agreed: an image that has at least some non-blank parts should be found correctly.

What about .exact()?

[skeets (skeets-malloy)]

Not sure we can use .exact() correct? Based on my info above? We need to be able to find the blank image at a specific location and if we only find one image at a location (of possibly many locations) then it may not be the image at the location we are looking for. Make sense?

[RaiMan (raimund-hocke)]

@skeets
no problem.

I will try it myself, since I am interested to fully understand the effect and look, wether it could be corrected somehow.

I will let you know about my results.

[skeets (skeets-malloy)]

Awesome thanks for the help!

FYI, the workaround is working, a blank child image with some "extra stuff" is found every time and where we expect it to be within the parent image using the findall method.

[RaiMan (raimund-hocke)]

@skeets
Some first tests with findAll() tell me:
- exact() does not change the behavior
- a blank image is not found in all possible/expected locations and subsequent matches overlap

I will have a deeper look into the implementation, but I doubt, that I can really change something (most of it is still baken in C++ deep down in the inherited hell from the time I took over ;-)

... and I will check what my preflight version 2 is doing here.

Thanks for finding, reporting and feedback.

BTW: what version of SikuliX do you use?

[skeets (skeets-malloy)]

OK cool glad it's not just me. :P Umm we are using 1.1.1.

Sikuli Version: SikuliX 1.1.1
OS Version: win 6.1
Java Version: 8
Python Version: 2.7.0

[RaiMan (raimund-hocke)]

@skeets
thanks. fine.
Now I am sure, that I see what you see ;-)

I will keep you informed.

[RaiMan (raimund-hocke)]

@skeets
Ok, with version 1.1.1+ I have to leave you with the workaround for now.
Did not find any possibility to change the behavior.

It is on the list for version 2.