[1.1.0] Image recognition doesn't work with Google Chrome 55 on Debian Jessie --- caused by problems with java.awt.Robot

[Roberto Fasciolo]

After upgrading Chrome to 55.0.2883.75-1 (the latest available stable) on my Debian Jessie laptop image recognition stopped working. I tried to capture the screen with Screen.capture() and the area where the web page should have been appeared as completely black there (while other screenshot programs like scrot or gimp captured the screen correctly).

Sikuli version is 1.1.0 and the OS is Debian Jessie 64 bit.

Reverting back to Chrome 54 fixed the issue.

[RaiMan (raimund-hocke)]

already checked with SikuliX 1.1.1?

multimonitor setup?

What Java version?

[Roberto Fasciolo (rob-2)]

Didn't yet check with 1.1.1, will do it as soon as possible.

I'm using Oracle Java 1.8.0_112 and when tested I was using only the laptop screen. To be precise when using Screen.capture() I could see in the image the Mate desktop's panels and Chrome's title bar (it's set to use the system title bar, not Chrome's own). So I suspect it's something related to the way Chrome renders it's window content rather than Sikuli itself, something that changed between version 54 and 55.

[RaiMan (raimund-hocke)]

thanks for the information.

I agree, that this has something to do with the window/GUI system and how Chrome uses it.
Inside the SikuliX implementation the screen capture simply uses Java's Robot class, which does not cover all screen/window/GUI setups (e.g. some menubar implementations, full screen situations, ...).

I checked on Mac and had no issues.

I will check on Ubuntu the next days.

[Roberto Fasciolo (rob-2)]

I made more tests and both Sikuli 1.1.0 and Sikuli 1.1.1 have the same issue (sometimes a bit of the web page is visible but in most of the cases the screen is plain black).

One thing I noticed is that if I run the script against a Chrome window I've opened myself directly from the Mate panel on top everything always works fine, the issue happens only when I open the window by changing persona the issue appears. And none of this happens with Chrome 54, with that version everything works fine.

[Roberto Fasciolo (rob-2)]

Replicated the issue on another laptop by using directly java.awt.Robot. So I would say it's not a Sikuli issue.

[Roberto Fasciolo (rob-2)]

Reported as Chrome issue: https://bugs.chromium.org/p/chromium/issues/detail?id=671565

[RaiMan (raimund-hocke)]

Wouw - great work. Thank you.