screenshots not recognised when re-running script

[HaileyN]

After writing a script and ensuring that all the regions chosen are recognised and correct I run the script and it works. however after re-opening the script and re-running it some of the regions are no longer found (either finds no matches or it finds completely different regions) yet the screen being referenced remains exactly the same.

Using Sikuli version 0.9.8
Windows XP 32-bit

I would really like to use this app for work, I think the idea is great, but this is a problem as if I need to run the script more than once i keep having to go back in and edit the script in some way or another even though there have been no changes to the screen that is being referenced. and it seems like I might be forced to look for another alternative

[RaiMan (raimund-hocke)]

please post your script here and the content of the message area in a bad run. Some info on the dimension of the images or a screenshot as attachement would be helpful.

possible reason: I found that Sikuli has severe problems with images, that exceed certain dimensions and contain much text, e.g. complete app windows (both Mac and Win). I'm preparing a bug report on that.

to test, just script:

switchApp("SomeOtherApp") # to be sure Sikuli IDE is out of way
find(capture())

no parm! when run you will be turned into the interactive capture mode (like with the foto symbol in the IDE) and then Sikuli tries to find what you just captured. You will be surprised about the results

By the way: You say "ensuring that all the regions chosen are recognised and correct". Do you use the Preview (click on an image and check the result in the little preview window)? If you play around with slider and number, and click OK, the image is changed into a pattern object pattern(<image>).similar(decimal).firstN(n). Info if used and where.

[HaileyN (hailey-niemand)]

Here is the script

find( )
click( )
type("a", KEY_CTRL)
type("100")
find( )
click( )
click( )
click( )
click( )
type("a", KEY_CTRL)
type("100")
click( )
click( )
click( )
click( )
hover( )
click( )
wait( )
click( )
click( )
wait( )
click( )
click( )
click( )
click( )

Sikuli is out of the way as I have in on my second monitor and I am using the primary monitor to work on and capture from as I know that Sikuli doesn't recognise the second monitor.

Also, Yes, I use the preview to ensure that the regions selected are correct. I have had to do this at multiple stages/on multiple lines, at times I have had to do this at for every step. The problem is that after doing this on the first run the script will work, but on re-running the script it no longer finds the correct regions and often the regions that are found seem to be completely random. The result is that I have to completely rewrite the script and it will work, but then after 1 or 2 runs it no longer works correctly.

[RaiMan (raimund-hocke)]

My experiences taken into account, I can give you the following hints:

I believe, they have a bug in Sikuli, witch matching images, that exceed either in width or heigth a value of 200 pixel. I find, that something with the dims [w, h]
[max-screenwidth, 200] or [200, max-screenheight]
works perfectly
e.g. [300, 300] doesn't work
My hint: try to capture slices. try out your max. dims - may be more than 200 - but on my screen 1920 x 1200 the 200 limit works.

Second point is about timing and control: with your scripts, you have the situation, that a click() e.g. works or not, you only see the effect visually: works or from some point onwards it stops working correctly.

If I develop a script that should work under all conditions, I implement at least some timing with sleep(x), to assure, that the screen has changed the way I expect.

Another option is to do something like this:
match = wait(<image>)
if match:
    click(match)

This waits for the appearance of what you want to click (3 seconds max in standard). It gives back None if the timeout happens and otherwise the match, which in turn you can click. If None, nothing happens.
Since we are using python syntax, take care, that every following statement, that depends on match not being None is indented correctly.

if you store your waits and finds in variables, you can use them later on to make decisions.
m1 = wait()
if m1:
    click(m1)
    ... do something else

else:
    popup("Sorry!")

if m1:
    m2 = wait()
    if m2:
    ... do sthg.

if m2:
   ... and so on

with this repeated if's you have only 2 indentation levels and a good overview over the dependencies.

I made a test taking care of the recommendations above and a script with 35 commands worked many times with no problems, even as exported executable.

[HaileyN (hailey-niemand)]

Ok, but in some cases the image is no larger than 200 x 200 and it will work the first time it is run it works fine, but if i close the app and reopen and try run the script again it won't find the regions it was previously finding (for example when asked to click a "save" button and instead it will find a "get approval" button)

it just seems odd that if it finds something initially and then on running the script later no longer finds this side thing when nothing has changed in the script or on the screen

[RaiMan (raimund-hocke)]

Ok, I admit, there is something weird going on.

tried this simple case:

find(capture())
find(capture())

it lets you interactively capture something. I always capture the same area (ok, 1 or 2 pixels +/- ;-)

When I start the IDE at least one of them does not find anything. I had cases, where both gave None.

Then after 2 or 3 reruns, the script finds in both cases. Thats what you would expect from the beginning.

After starting the IDE again and work from scratch (I do not even save anything), same happens.

This is a bug.

[RaiMan (raimund-hocke)]

seems to be solved with version 0.10