find() not working with much text in region

[RaiMan]

OS X 10.6.2 Sik 098

based on some tests I made together with bug (screenshots not recognised when re-running script) https://bugs.launchpad.net/sikuli/+bug/520524 I found 2 "find() not working"-bugs, that I will document.

This is the first one: Too much text in a capture
If a captured region contains significantly more text, than grafical information, the find may not work at all or with a lower similarity will find a displaced area. This can be reproduced with all the capture techniques available. For documentation I have chosen one, that does not depend on manual capturing (either in advance or interactively) to exactly know the (x, y, w, h).

The material is available at:
- script together with messages: http://files.me.com/rhocke/p0evyc
- a screenshot as an overview: http://files.me.com/rhocke/92eafj

This is the script I use:
switchApp("iPhoto")
sub = <captured subregion> # you have to capture a region
pos = (sub.x, sub.y, sub.w, sub.h)
(x, y, w, h) = pos
print "++++ Our testarea: ", pos
p = capture(x, y, w, h)
print "++++ find all ", find(p)
for iW in range (20, w, 50):
   print "++++ Try with width = ", iW
   p = capture(x, y, iW, h)
   fnd = find(p)
   print "++++ found? ", fnd
   if fnd and iW > 200:
      break
   if not fnd:
      pat = Pattern(p).similar(0.2).firstN(1)
      print "++++ try again with Pattern ", pat
      fnd = find(pat)
      print "++++ found pattern? ", fnd

To really understand, what happens, look at the screenshot in parallel.
The critical area is the menu section below "MobileMe Gallery" in the left column of iPhoto. If I capture only the menu section (about 160 x 180 in size), this is not found. If you reduce the similarity to 0.29, it is found, but displaced by about (-20, -100).

How my test works:
First I capture a region from left side with the mobileme-icons to the edge of the screen on the right. Then I start to capture slices starting with width 20 and stepping 50 always starting left side of the region.
You can see:
- the column of icons is found
- a slice about 120 in width is found, but not the slice with 170
- starting with a width of about 350, its found always, so I stop at the first successful find above 200.

when not found, I reduce the similarity to 0.2 and try again with this pattern. That is always successful here (must not be in other situations), but with a significant displacement of the match area (compare the pattern match coordinates with the region).

interesting is the fact, that a single menu entry like "Feuer_Hofhausstraße" together with the mobileme-icon is always found. This gave me the idea, that it has something to do with the dimension of the images to find (another weird situation I found and will document in another bug report)

[RaiMan (raimund-hocke)]

One more thing

As you will see, when testing other situations, for one of these find()'s that are not successful, there are up to 4 screen captures reported. Normally its only one, which is what one expect.

[RaiMan (raimund-hocke)]

in the comment above I stated:

As you will see, when testing other situations, for one of these find()'s that are not successful, there are up to 4 screen captures reported. Normally its only one, which is what one expect.
I found the reason for that myself:

A find() in the standard is waiting max 3 seconds for the pattern to come up on the screen. This is an implicit wait(). If you know what should be on the screen, it boosts performance if you use

setAutoWaitTimeout(1)

that hinders find() after a not-found to wait for some aliens to help the screen to come up with the image we are looking for.

Normally you would think, that setAutoWaitTimeout(0) would be the way to say "don't wait at all", but there seems to be a bug, because if you use 0, the <try: except:> does not work any longer.

See full activity log

[RaiMan (raimund-hocke)]

version 0.10: fast and precise now.