[request] find/findAll to accept list of patterns

[anatoly techtonik]

I wonder if it is possible to create effective algorithm to search for multiple patterns during single scan. I know this is possible for sure if similarity is 1.0. Anyway, it would be nice to have this ability for findAll function, where you need to find a lot of different images in one pass. Using `for image in list_of_patterns: stack += findAll(image)` is very slow.

[RaiMan (raimund-hocke)]

until this feature might be available:
use python threading and dispatch each find/findAll in a thread, so they run in parallel.

[Greg Toombs (greg-toombs)]

The threading workaround is reasonable, until you find the one element you're looking for, and Python has no support for killing the threads that keep looking for non-existent elements, and the CPU is pinned at 100%.

[RaiMan (raimund-hocke)]

Thanks again for the challenge ;-)

Of course this is possible in Python/Jython ...
... and here is a rather basic solution

you can:
- add as many images to be searched in parallel
- give each one an individual wait time
- stop all running searches, if you want at any time

import thread

# the threadable find function
def tfind(parms):
    global shouldStop
    end = time.time()+parms.get("waittime", 3) # standard wait time 3 seconds
    reg = parms.get("region", SCREEN) # standard region whole screen
    parms["match"] = None
    parms["finished"] = False
    while (not shouldStop) and time.time()<end:
        if reg.exists(parms["pattern"], 0):
            parms["match"] = reg.getLastMatch()
            break
    parms["finished"] = True

# setup a pattern list to be paralleled
pats = []
# model pattern: {"pattern": some image or pattern, "waittime": seconds, "region": some region}
pats.append({"pattern":"1352549030079.png"})
pats.append({"pattern":"1352551675237.png", "waittime":10})
pats.append({"pattern":"1352551662476.png"})

# start the threads
shouldStop = False
for p in pats:
    thread.start_new_thread(tfind, (p,))

#wait for termination
end = time.time()+5 # search max 5 seconds
while True:
    # stop searches if global search time elapsed
    if time.time() > end:
        shouldStop = True
        print "search stopped"
        wait(2) # give time to the threads to stop
        break
    # wait until all threads have terminated
    ended = True
    for p in pats:
        ended = ended and (p.get("finished", False))
    if ended:
        print "search ended normally"
        break

# check results
for p in pats:
    if p["match"]: p["match"].highlight(1)
    else: print "not found:", p["pattern"]

Happy scripting ;-)

I will implement this in the Sikuli API as something like that:

findAny(pattern, pattern, ..., waittime) # comes back when the first one is found
findAny(listOfPatterns, waittime)

findAll(pattern, pattern, ..., waittime) # comes back after all are found
findAll(listOfPatterns, waittime)

both come back latest after wait time and return a list of matches corresponding positionally to the given patterns

[RaiMan (raimund-hocke)]

To make this even more flexible, the Pattern class should have more attributes:

- defaults:
count = 1
waittime = standard-waiting-time

- count
... defines how many images should be searched in one run
setting count = 0 would return as many matches as possible within the given wait time

- waittime
... a specific waittime for this pattern, that should be used with find() (overwritten when using wait())

usage:
p = Pattern("some-image.png").setCount(5).setWait(10)
matches = find(p) # returns a list of matches like findAll()

[Greg Toombs (greg-toombs)]

Brilliant. Do you need help testing this? How would I obtain and install a test release?

[Greg Toombs (greg-toombs)]

Could I suggest that, instead of a polling loop (increasing the CPU burden even more), use a threading.Event shared between all of the threads, and have the main thread block on it until one of the threads completes and sets the event?

[RaiMan (raimund-hocke)]

Of course will this be implemented event based on the Java level.

The polling loop above was just a quick and dirty solution to implement this with basic Jython features.
It can be smoothed a little, by putting a wait(1) before the next poll starts.

But somewhere inside will always be some polling loops. And the resulting cpu usage depends more on the number of parallel searches (which in turn are search loops with pauses (WaitScanRate)), then on the fact, that we are polling.

Beginning next week I will upload test versions.

[Greg Toombs (greg-toombs)]

Awesome! :D