[request] want a feature to select an image in a drop down list

[obiwan-92]

Hello.

It's probably one of the most difficult thing to do with Sikuli.
I have no solution at all to generalize this function to work every time.
But I think it will be more than useful.

-- system --
Sikuli 1.0.1
Win 7 (32)
Java 1.7.51

Regards.

[obiwan-92 (obiwan-92)]

Apparently, I forgot to write it but it's a request.

[RaiMan (raimund-hocke)]

fully agreed ;-)

But since I know Sikuli (late 2010) I never had a good idea, how to support it better.

Therefor I always recommend, to try all mouse/keyboard options, before using images.

So help me with good ideas about enhancements of the Sikuli features, to support drop-down-menus better.

[obiwan-92 (obiwan-92)]

Hello.

Well, maybe we can do it like this :

# It's just a suggestion
def selectDropList(img1, img2):
    click(img1) # Open the drop down list
    reg = some function to find the region where some pixels change, corresponding where the drop down list
    while not reg.exists(img2): # Put the time-out to 0 : don't wait each time
        type(Key.PAGE_DOWN)
        if reg.change(): # check if some pixel change in the region
            return False # we reach the end of the list without find the item (img2)
    click(img2)
    return True # we find and click on the item

The most difficult thing to do for me, is to determine the region of the drop down list.
The second difficult point, is to create a function to see if some pixels change in a region, I suppose we can do it with the observe methods.

Hope it's help.
Regards.

[RaiMan (raimund-hocke)]

Ok, thanks for the input ...
... but this is not really what I expected.

At least, this could help as a general guideline to approach an individual solution (based on your suggestion):

def selectInDropDown(img1, img2):
# img1 need to have the width of the drop down opening below
# img2 is a capture of an element in the list
    ddhead = exists(img1)
    if not ddhead: return False # dd head not found
    ddlist = ddhead.below() # assuming list opens below
    img3 = capture(ddlist)
    click(ddhead)
    ddlist.waitVanish(img3) # wait for list to open
    while not ddlist.exists( img2, 0):
         img4 = capture(ddlist)
         type(Key.PAGE_DOWN); wait(1)
         if (delist.exists(img4, 0): return False // end of list reached
     click ddlist.getLastMatch()
     return True

What I am after is some Pattern like definition of the list physics and then you can say:

dd = DDPattern(....)
ret = click(DDPattern.use(img2))
if ret = .....

[obiwan-92 (obiwan-92)]

Hum...
So your DDPattern will return some sort of capture to your list ?
But how did you manage when you have elevator ?

[RaiMan (raimund-hocke)]

not really.
DDPattern holds all information, that defines the drop down (DD):
- base image (to be found and clicked to open the DD)
- orientation
- navigation supported

... the IDE could have some support like with Preview:
- you capture the image and open preview and there the tab DD (new ;-)
- you select some options
- you might test the DD and capture the DD to
- define the geometry (entry heights)
- ....

on save this will create the respective Jython elements in the code (like with the current Pattern().similar.targetOffset())

so the usage would be (corrected)

dd = DDPattern(<the defining parameters created by preview>)
ret = click(dd.use(img2))
if ret = .....

[Ryan (ryan-pisano)]

Why not dynamically create a region based on the corners of the dropdown?

It wouldn't matter what direction the dropdown opens since the user will specify what the top left, bottom left, and top right region look like. If you capture the images carefully it won't matter what is behind the dropdown or how many options are in the dropdown (important in most of my scenarios). If all actions relating to the dropdown utilize this region it will also allow for separate scoping of both up/down/scroll arrows and any text searches by OCR.

The code is pretty simple to abstract into a function, or superclass depending on how Object Oriented the project is, and re-use all over the project. I feel like from an OO perspective that it seems a little domain specific to be built into sikuli as a class of some kind, but as a function the logic would be as simple as (psuedo code):

openDropDown(ddHead, topLeft, topRight, bottomLeft):
    click the head
    find each image
    calculate the width from topLeft to topRight
    calculate the height from topLeft to bottomLeft
    region(topLeft.getX(), topLeft.getY(), width, height)

Here the most general case would be to return that new region, however if it were part of a class it could be set as an attribute and then provide scrolling, clicking, and reading options.