[2.0.0] Windows 10: Move mouse for click not working since new JRE and Windows update with Java 8 and HiDPI --- workaround Java 9 and scaling 100% OR try with Java 11

[RaiMan]

see: https://github.com/RaiMan/SikuliX1/issues/27
--------------------------------------------------
********************* Java problem
seems to be related to a problem with HiDPI and scaled monitor setups
https://bugs.openjdk.java.net/browse/JDK-8196030

The problem seems to exist with Java 8 not supporting HiDPI.

Using Java 9 or 10 on HiDPI systems with a scaling of 100% seems to work

Using Java 11 these experiences are reported:
early build of jdk 11 and sikuli 1.1.4 is working good for hidpi, but still we observed few issues mentioned below:

1) Sikuli scripts written using 4k (3840*2160) with recommended scaling (300%) is working only for the same recommended settings. If we change the resolution to 2K or 1080P or below , and also if we change scaling, scripts are failing

2) Sikuli scripts written on 2k (2560*1600) with recomended scaling (200%) is working for for resolutions below 2k like 800*600 to 1920*1080 (Downscaling is working with this) and for 4k it is not working (Upscaling is not working)
-------------------------------------------------

wondering if anyone else is seeing this.

I install the latest big update for Windows this morning and also accepted an update to the latest JRE.

Now my when Sikuli moves the mouse to click on a match it doesn't land on the coordinates. It even recognizes that it missed and has an error message saying it tried to move the mouse to (x, y), but landed on (p, q) instead.

Any idea what is happening and how to fix it?

Thanks!

[RaiMan (raimund-hocke)]

attached this is a test jar

usage:
java -jar MoveMouseAround.jar loops delay

loops and delay are optional integers (positional parameters):

loops: how often should the mouse be moved randomly inside the area (margin of 200 all sides inside screen), default: 5

delay: how many milliseconds should java.awt.Robot wait after a move, default 20 (max 60)

the output is something like this:
[MoveMouseAround] Using: loops = 5; Robot.autoDelay = 20
[MoveMouseAround] GraphicsEnvironment: 1 ScreenDevices
[MoveMouseAround] Screen: (0,0 1920x1080)
[MoveMouseAround] using area: (200,200 1520x680)
[MoveMouseAround] *** visitCorners of (200,200 1520x680)
[MoveMouseAround] Moved to: (200,200)
[MoveMouseAround] Moved to: (1720,200)
[MoveMouseAround] Moved to: (1720,880)
[MoveMouseAround] Moved to: (200,880)
[MoveMouseAround] Moved to: (200,200)
[MoveMouseAround] *** movingAround in (200,200 1520x680) for 5 times
[MoveMouseAround] Moved to: (1225,347)
[MoveMouseAround] Moved to: (741,717)
[MoveMouseAround] Moved to: (1714,546)
[MoveMouseAround] Moved to: (1670,272)
[MoveMouseAround] Moved to: (1656,424)

if a move is not correct:
[MoveMouseAround] Moved to: (...,...) but is: (..., ...)

[RaiMan (raimund-hocke)]

... and here the source if someone is interested.

[Dan Quimby (daqu40)]

Thanks RaiMan. I'll give this a try. I suspected there might be something in the scaling. I fiddle with it a bit, but no success.

[sdraganov (sdraganov)]

Is there workaround for this bug?
Can't find Java 9 at www.oracle.com - should I download it from third party website?

[RaiMan (raimund-hocke)]

@sdraganov: Try with Java 10

[sdraganov (sdraganov)]

I tried that already but it gives me the following error when running SikuliX 1.1.2
https://pastebin.com/dWerkpTa

Will try with the nightly 1.1.3 version.

[sdraganov (sdraganov)]

Seems to work fine with 1.1.3, at least with very simple script, just a single click command.
Will do more tests and will keep you posted in case something goes wrong.

[Sachin (sachin.kademane)]

Hi @Raiman,

we are using sikuli 1.1.3 with java 9.0.4 and jython 2.7.1...Sikuli is unable to detect images in 4k resolution even when scripted on same 4k panel..what else i need to try?

[RaiMan (raimund-hocke)]

as mentioned in the bug-title:
try with Java 10.

I am sorry, I cannot help you:
- I do not have a 4K display (Mac Retina works)
- this is definitely a Java problem, as you can see in the net looking for "Java hiDPI"

[Michelle (mnovabox)]

Hi @Raiman,

So I have the same error:

[error] RobotDesktop: checkMousePosition: should be L(3520,540)@S(1)[2560,0 1920x1080]
but after move is L(3504,557)@S(1)[2560,0 1920x1080]
Possible cause in case you did not touch the mouse while script was running:
 Mouse actions are blocked generally or by the frontmost application.
You might try to run the SikuliX stuff as admin.

Now I switched to java 11.
Now I get the following error:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/C:/jython2.7.0/jython.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.PySystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

This is a known problem for Java 9/10/11 and the developers of jython don't know how to fix it:
https://stackoverflow.com/questions/46230413/jdk9-an-illegal-reflective-access-operation-has-occurred-org-python-core-pysys

How did you bypass this?

[RaiMan (raimund-hocke)]

@Michelle
your question is not related to this bug topic !

see bug 1749262

[Michelle (mnovabox)]

ok thanks

[RaiMan (raimund-hocke)]

@garimapriya89
Please do not add comments containing questions to other questions/bugs, that are not definitely about the same context.
Always start with your own new question.

your problem:
before working against some webdriver, you should make basic experiences against the real screen with a real browser.

your snippet:
Screen screen = new Screen();
//Thread.sleep(3000);
//Debug.info("Screen: %screen", screen);

ImagePath.setBundlePath("C:\\Users\\gp0924\\Desktop\\LatestSikuliImages");
String img = "About.PNG";
Match match = screen.exists(img);
if (null != match) {
  match.highlight(); //already logs the match
  match.click(); // do not search again
} else ;//not found