Autopilot uses mouse events, puts device in windowed mode

Bug #1471598 reported by Christian Dywan on 2015-07-05
This bug affects 3 people
Affects Status Importance Assigned to Milestone
autopilot (Ubuntu)
unity8 (Ubuntu)

Bug Description

AP tests emulating mouse to run, which puts the device into windowed mode.
mouse presence is used to toggle the windowed mode on mobile devices.

----orig bug ----
The autopilot tests on Krillin run in windowed mode. I don't think this should ever happen on that form factor.

See also a bunch of dubious errors below.

Tests running...
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<mir::socket_disconnected_error> >'
what(): Failed to send message to server: Broken pipe
Failed to connect to server. Error was :Failed to connect to server socket: Connection refused

(process:5172): GLib-GIO-CRITICAL **: Error while sending AddMatch() message: The connection is closed
g_dbus_connection_real_closed: Remote peer vanished with error: Underlying GIOStream returned 0 bytes on an async read (g-io-error-quark, 0). Exiting.
/bin/bash: line 1: 5172 Terminated SUPPRESS_DEPRECATED_NOTE=yes autopilot3 run ubuntuuitoolkit

Related branches

Zoltan Balogh (bzoltan) wrote :

the same happens with any other tests

affects: ubuntu-ui-toolkit (Ubuntu) → unity8 (Ubuntu)
Changed in unity8 (Ubuntu):
status: New → Confirmed
kevin gunn (kgunn72) on 2015-07-06
Changed in unity8 (Ubuntu):
importance: High → Low
Zoltan Balogh (bzoltan) wrote :

Why is it low?
It broke all autopilot tests. All. Not only the unity8 but all the UITK and all the project.

There is a workaround to locally force the Staged mode when tests are run, but for people who are validating MR with running AP tests it is far from obvious.

kevin gunn (kgunn72) wrote :

ok, i misunderstood - i thot you were running in windowed mode on purpose or something.

After some further discussion:
understanding is that AP is emulating mouse events
unfortunately for convergence, mouse events is precisely the trigger for windowed mode
so need to find a solution for AP that can work in the presence of converged code base.

Changed in unity8 (Ubuntu):
importance: Low → Medium
Changed in autopilot (Ubuntu):
importance: Undecided → High
Changed in unity8 (Ubuntu):
importance: Medium → High
Changed in autopilot (Ubuntu):
status: New → Confirmed
summary: - Autopilot tests on Krillin run in windowed mode
+ Autopilot uses mouse events, puts device in windowed mode
description: updated
Michael Zanetti (mzanetti) wrote :

Actually the requirement should be to enable autopilot tests to choose different input devices. For example some function method decoration to specify which types of input devices (touchscreens, keyboards, mice..) should be plugged for a particular test.

Michael Zanetti (mzanetti) wrote :

For now I'm changing the default setting in unity to stick to Staged mode.

Christopher Lee (veebers) wrote :

Digging into this its not 'Mouse' events triggering this (you cannot create a Mouse instance on a device without installing all the XLib stuff).
It's the creation of a Touch device that is triggering this. You can observe this by shelling into a device and starting python3 terminal:
  >>> from autopilot.input import Touch
  >>> mytouch = Touch.create()
  >>> mytouch.close()

(When you close the python terminal unity will revert back to its normal state.)

Looking at what happens in touch creation this is the *paraphrased* code that triggers the change to windowed:

  >>> from evdev import UInput, ecodes as e
  >>> mydev = UInput(events={e.EV_ABS: [(e.ABS_X, (0, 0, 0, 0)), (e.ABS_Y, (0, 0, 0, 0))], e.EV_KEY: [e.BTN_TOOL_FINGER]}, name='autopilot-finger', version=0x2, devnode='/dev/uinput')
  >>> mydev.close() # Unity8 will become un-windowed at this point.

The actual code where its created lives here[1] the interesting events part here[2] and there are more events passed in but this is enough to trigger the windowing in unity8.

My thoughts are that either autopilot is creating the touch device incorrectly or that the detection of a mouse device in unity8 is picking this up inadvertently.

I'm currently seeking clarification re: the touch device creation (note this comment in the source[3]).


Michael Zanetti (mzanetti) wrote :

Thanks for this information Chris. I've ran this while watching the debug output for unity8 and turns out the above you registers a device with the evdev property ID_INPUT_TOUCHPAD. A touchpad is considered a "desktoppy" thing too, hence switches to windowed mode too.

I have modified your example to this (dropping the BTN_TOOL_FINGER):

mydev = UInput(events={e.EV_ABS: [(e.ABS_X, (0, 0, 0, 0)), (e.ABS_Y, (0, 0, 0, 0))]}, name='autopilot-finger', version=0x2, devnode='/dev/uinput')

and it does not register the Touchpad any more, hence not switching to Windowed mode. Changing the line to this:

mydev = UInput(events={e.EV_ABS: [(e.ABS_X, (0, 0, 0, 0)), (e.ABS_Y, (0, 0, 0, 0))], e.EV_KEY: [e.BTN_TOUCH]}, name='autopilot-finger', version=0x2, devnode='/dev/uinput')

Makes it correctly register as a ID_INPUT_TOUCHSCREEN and does not make unity8 go into windowed mode.

So seems the proper fix is to use , instead of BTN_TOUCH instead of BTN_TOOL_FINGER in autopilot's Touch device.

Michael Zanetti (mzanetti) wrote :

And here's the code in AP:


def _get_touch_tool():
    # android uses BTN_TOOL_FINGER, whereas desktop uses BTN_TOUCH. I have
    # no idea why...
    if autopilot.platform.model() == 'Desktop':
        touch_tool = e.BTN_TOUCH
        touch_tool = e.BTN_TOOL_FINGER
    return touch_tool

I don't know either why android uses a touchpad (BTN_TOOL_FINGER), but looking at this code it seems all our tests use a touchscreen (BTN_TOUCH) when running on desktop already.

I've pushed a fix to: lp:~mzanetti/autopilot/dont-use-touchpad-for-touch

Changed in autopilot (Ubuntu):
status: Confirmed → Fix Released
Changed in unity8 (Ubuntu):
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers