Touchscreen controls both screens in dual-monitor setup

[Steve Magoun]

SRU Request:

Laptops and all-in-one systems with touchscreens don't currently map the touch input device onto the internal display. As a result, when the screen changes (e.g. the main output is rotated or an external output is connected), the touch input device maps onto the entire screen, regardless of the only relevant area, thus making the touchscreen completely useless.

[Impact]
 * The touchscreen becomes unusable if an external output is connected or if the main output is rotated.

[Test Case]
 * Either plug in an external monitor or rotate the internal monitor
    - Expected: the touchscreen should respond to touch as usual.
    - Bad behavior: the touchscreen is unusable and responds by selecting areas other than the one the user actually selects.

[Regression Potential]
 * Low, it only affects the main touchscreen (tablets or multiple touchscreens are not supported) and it only adds a feature that is currently missing.

[Other Info]
 * N/A

---

My laptop has a touchscreen. It works fine in 14.04 until I plug in an external monitor. When using an external monitor, input from the touchscreen is remapped so that the touchscreen provides input to both monitors. The result is that there is no longer a 1:1 correspondence between moving your finger and the cursor. This is confusing and difficult to use.

A touchscreen is a direct input device (vs an indirect one like a mouse/touchpad); input from the touchscreen should be bound to the physical display that it's attached to.

To reproduce:
1) On a computer with a touchscreen, use the touchscreen to move a window around, including to the edges of the screen. Note that the window moves exactly with your finger.
2) Plug in an external monitor
3) Try the same actions as in step 1

Expected results:
The touchscreen continues to operate as in step 1, allowing you to manipulate items on the display that contains the touchscreen. The touchscreen does not interact with windows, etc on the external monitor

Actual results:
The touchscreen is remapped across both displays. The result is that touch events no longer happen 'under the finger'. Assuming the two displays are the same size and resolution, moving your finger 1cm will cause the window or other objects to move 2cm onscreen. It becomes impossible to use the touchscreen to interact with widgets (menus, buttons, etc).

This is described in couple places:
http://askubuntu.com/questions/51445/how-do-i-calibrate-a-touchscreen-on-a-dual-monitor-system
http://askubuntu.com/questions/71768/touchscreen-and-additional-external-monitor

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: xinput 1.6.1-1
ProcVersionSignature: Ubuntu 3.13.0-14.34-generic 3.13.5
Uname: Linux 3.13.0-14-generic x86_64
.tmp.unity.support.test.0:

ApportVersion: 2.13.2-0ubuntu5
Architecture: amd64
CompizPlugins: [core,commands,composite,opengl,compiztoolbox,decor,vpswitch,snap,mousepoll,resize,place,move,wall,grid,regex,imgpng,session,gnomecompat,animation,fade,unitymtgrabhandles,workarounds,scale,expo,ezoom,unityshell]
CompositorRunning: compiz
CompositorUnredirectDriverBlacklist: '(nouveau|Intel).*Mesa 8.0'
CompositorUnredirectFSW: true
CurrentDesktop: Unity
Date: Mon Mar 3 14:41:14 2014
DistUpgraded: 2014-02-12 13:40:42,704 DEBUG enabling apt cron job
DistributionChannelDescriptor:
 # This is a distribution channel descriptor
 # For more information see http://wiki.ubuntu.com/DistributionChannelDescriptor
 canonical-oem-somerville-precise-amd64-20130203-1
DistroCodename: trusty
DistroVariant: ubuntu
DkmsStatus:
 oem-audio-hda-daily-lts-quantal, 0.201308192259~precise1, 3.5.0-45-generic, x86_64: installed
 virtualbox, 4.3.6, 3.13.0-12-generic, x86_64: installed
 virtualbox, 4.3.6, 3.13.0-14-generic, x86_64: installed
 virtualbox, 4.3.6, 3.13.0-8-generic, x86_64: installed
EcryptfsInUse: Yes
GraphicsCard:
 Intel Corporation Haswell-ULT Integrated Graphics Controller [8086:0a16] (rev 09) (prog-if 00 [VGA controller])
   Subsystem: Dell Device [1028:060a]
InstallationDate: Installed on 2013-12-02 (90 days ago)
InstallationMedia: Ubuntu 12.04 "Precise" - Build amd64 LIVE Binary 20130203-13:50
MachineType: Dell Inc. XPS13 9333
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.13.0-14-generic root=UUID=0b9db31c-747b-40ad-bbbd-13a9a29caece ro quiet splash vt.handoff=7
SourcePackage: xinput
UpgradeStatus: Upgraded to trusty on 2014-02-12 (19 days ago)
dmi.bios.date: 11/11/2013
dmi.bios.vendor: Dell Inc.
dmi.bios.version: A01
dmi.board.name: 0GFTRT
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 8
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: 0.1
dmi.modalias: dmi:bvnDellInc.:bvrA01:bd11/11/2013:svnDellInc.:pnXPS139333:pvr:rvnDellInc.:rn0GFTRT:rvrA00:cvnDellInc.:ct8:cvr0.1:
dmi.product.name: XPS13 9333
dmi.sys.vendor: Dell Inc.
version.compiz: compiz 1:0.9.11+14.04.20140218-0ubuntu1
version.ia32-libs: ia32-libs N/A
version.libdrm2: libdrm2 2.4.52-1
version.libgl1-mesa-dri: libgl1-mesa-dri 10.1.0~rc1-1ubuntu4
version.libgl1-mesa-dri-experimental: libgl1-mesa-dri-experimental N/A
version.libgl1-mesa-glx: libgl1-mesa-glx 10.1.0~rc1-1ubuntu4
version.xserver-xorg-core: xserver-xorg-core 2:1.15.0-1ubuntu6
version.xserver-xorg-input-evdev: xserver-xorg-input-evdev 1:2.8.2-1ubuntu2
version.xserver-xorg-video-ati: xserver-xorg-video-ati 1:7.3.0-1ubuntu3
version.xserver-xorg-video-intel: xserver-xorg-video-intel 2:2.99.910-0ubuntu1
version.xserver-xorg-video-nouveau: xserver-xorg-video-nouveau 1:1.0.10-1ubuntu2
xserver.bootTime: Mon Mar 3 11:02:50 2014
xserver.configfile: default
xserver.errors:

xserver.logfile: /var/log/Xorg.0.log
xserver.outputs:
 product id 4933
 vendor CMN
xserver.version: 2:1.15.0-1ubuntu6

[Maarten Lankhorst (mlankhorst)]

Did this work correctly in 13.10?

[Steve Magoun (smagoun)]

@Maarten: I did not try 13.10, but I did try in 12.04. It was broken in 12.04 too.

[Alberto Milone (albertomilone)]

@Steve: please attach the output of the following commands:

xinput list

Then look at the output and see what id matches your touchscreen (e.g. id=12), and finally type:

xinput list-props $the_id_you_found

[Steve Magoun (smagoun)]

@Alberto:

steve@aether:~$ xinput list
⎡ Virtual core pointer id=2 [master pointer (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ SYNAPTICS Synaptics Large Touch Screen id=12 [slave pointer (2)]
⎜ ↳ SynPS/2 Synaptics TouchPad id=15 [slave pointer (2)]
⎣ Virtual core keyboard id=3 [master keyboard (2)]
    ↳ Virtual core XTEST keyboard id=5 [slave keyboard (3)]
    ↳ Power Button id=6 [slave keyboard (3)]
    ↳ Video Bus id=7 [slave keyboard (3)]
    ↳ Power Button id=8 [slave keyboard (3)]
    ↳ Integrated_Webcam_HD id=13 [slave keyboard (3)]
    ↳ AT Translated Set 2 keyboard id=14 [slave keyboard (3)]
    ↳ Dell WMI hotkeys id=16 [slave keyboard (3)]
    ↳ Integrated_Webcam_HD id=9 [slave keyboard (3)]

steve@aether:~$ xinput list-props 12
Device 'SYNAPTICS Synaptics Large Touch Screen':
 Device Enabled (135): 1
 Coordinate Transformation Matrix (137): 1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
 Device Accel Profile (260): 0
 Device Accel Constant Deceleration (261): 1.000000
 Device Accel Adaptive Deceleration (262): 1.000000
 Device Accel Velocity Scaling (263): 10.000000
 Device Product ID (252): 1739, 2808
 Device Node (253): "/dev/input/event12"
 Evdev Axis Inversion (264): 0, 0
 Evdev Axis Calibration (265): <no items>
 Evdev Axes Swap (266): 0
 Axis Labels (267): "Abs MT Position X" (283), "Abs MT Position Y" (284), "None" (0), "None" (0)
 Button Labels (268): "Button Unknown" (256), "Button Unknown" (256), "Button Unknown" (256), "Button Wheel Up" (141), "Button Wheel Down" (142)
 Evdev Middle Button Emulation (269): 0
 Evdev Middle Button Timeout (270): 50
 Evdev Third Button Emulation (271): 0
 Evdev Third Button Emulation Timeout (272): 1000
 Evdev Third Button Emulation Button (273): 3
 Evdev Third Button Emulation Threshold (274): 20
 Evdev Wheel Emulation (275): 0
 Evdev Wheel Emulation Axes (276): 0, 0, 4, 5
 Evdev Wheel Emulation Inertia (277): 10
 Evdev Wheel Emulation Timeout (278): 200
 Evdev Wheel Emulation Button (279): 4
 Evdev Drag Lock Buttons (280): 0

[Ara Pulido (ara)]

Seb wrote in an email:

"In fact discussing a bit on IRC and looking at
https://bugzilla.gnome.org/show_bug.cgi?id=698284 it might not be as easy

Xorg/the hardware doesn't give us a way to determine that an input
system is linked to a screen. It's easy enough when you have 1 of each
to match them, it's less trivial if you have e.g 2 screens and an input
device. We can probably come up with some heuristics for common cases
(like assuming that if you have a laptop screen and an external monitor
and a touch input, that's probably the laptop screen one) but not sure
how well that's going to work..."

[Ara Pulido (ara)]

Related discussion in #ubuntu-desktop

18:24 < seb128> "we don't map touchscreens to an output, we map it to a portion of the desktop."
18:24 < seb128> " don't know how to fix this other than a database in the style of libwacom"
18:24 < tjaalton> that doesn't sound too promising..
18:24 < mlankhorst> probably needs to happen :/
18:26 < bregma> big problem with a generalized solution to that touchscreen problem is that there is no association between the touch device and the display
18:26 < mlankhorst> yeah..
18:27 < bregma> not even at the kernel level
18:28 < mlankhorst> but for now assuming touchscreen maps to the eDP or integrated screen would be best bet :/
18:29 < bregma> it would makea reasonable starting default but isn't anything like a good solution since there are lots of external touchscreens out there
18:30 < seb128> yeah
18:31 < mlankhorst> how are they handled on other platforms?
18:31 < seb128> it would also mean that if I dock my 3 years non-touch laptop on an external touch monitor things would be the wrong way around
18:31 < tjaalton> but confirmed..
18:31 < tjaalton> *bug
18:31 < mlankhorst> tjaalton: which?
18:31 < tjaalton> the touch one
18:32 < mlankhorst> ah
18:32 < tjaalton> i've got this lovely harris beach sdp..
18:32 < bregma> mlankhorst, how are they handled on other platforms? poorly, just like high-DPI
18:32 < tjaalton> now excuse me while it dist-upgrades trusty->trusty for the next two hours..
18:33 < tjaalton> hmm I have win8 on this too
18:35 < mlankhorst> could we make it a ui thing perhaps?
18:36 < mlankhorst> let the user suffer for next few years until things standardize :/

[Alberto Milone (albertomilone)]

Just as an update, I'm working on some code that will detect the touch screen, couple it with the laptop's screen, and map the touch input device onto the output whenever the screen configuration changes (e.g. on rotations, when new outputs become available, etc.). This won't require any user interaction.

[Alberto Milone (albertomilone)]

I already have a prototype that solves the use case mentioned in this bug report.

Apparently upstream also has code that fixes the same issue and covers more use cases. It doesn't seem like a small amount of code though. For this reason I'll take some time to see if a backport is possible or recommended over using my current implementation.

[Alberto Milone (albertomilone)]

I've gone through all the relevant upstream commits, and here are my conclusions:

1) The upstream code covers the use case of Wacom tablets with built-in displays (which, although useful, is not the use case discussed in this bug report). It would be way too much code to backport, especially for an SRU.

2) The upstream code covers the use case mentioned in the original description of this bug report, but I think backporting the new GsdDeviceMapper (a tool to couple input devices with displays) to unity-settings-daemon would be an overkill, just for this single use case. This said, introducing the GsdDeviceMapper is something definitely worth exploring, as a long term plan.

3) I can provide a fix for the use case in the original description with a much smaller amount of code. For this reason I'm going to complete and integrate my current prototype with the unity-settings-daemon. The result will be effective and good for an SRU.

[Alberto Milone (albertomilone)]

I've built packages for both i386 and amd64 with my fix. You will find them here:

http://people.canonical.com/~amilone/lp1287341/

Please install the package for your architecture, then log out, log back in, and try to reproduce the problem.

It would be interesting to test my fix on laptops with a touchscreen and on all-in-one desktop systems that have a touchscreen.

[Steve Magoun (smagoun)]

@Alberto: The package in http://people.canonical.com/~amilone/lp1287341/ fixes the behavior of the touchscreen for me. The touschreen now works as expected with and without an external monitor. Tested on a Dell XPS13 with factory-installed touchscreen.

[Ove Risberg (ove-risberg)]

Thanks!

It works for me :)

Lenovo Yoga 2 Pro with 32-bit Ubuntu 14.04.

[AceLan Kao (acelankao)]

Hi, I tested it on Dell OptiPlex 9030 AIO with 64-bit Ubuntu 14.04, but it doesn't work.
The touchscreen remaps to the whole extended screen.
I tried to re-login, reboot the system, and re-plug the hdmi cable, the symptom is the same.

ii unity-settings-daemon 14.04.0+14.04.20140414-0ubuntu1.1 amd64 daemon handling the Unity session settings

[Alberto Milone (albertomilone)]

@Acelan: please download the following binary, run it on that system, and attach the output:

http://people.canonical.com/~amilone/simple-xinput

Also please attach your /var/log/Xorg.0.log and the output of the "xinput list --long" command.

Note: in this case there's no need to reproduce the problem before collecting the data, this is just to see what type of touchscreen we're dealing with.

[AceLan Kao (acelankao)]

Here comes the logs

[Alberto Milone (albertomilone)]

ok, what happens if you type:

killall unity-settings-daemon

unity-settings-daemon --debug &> mylog.log

then reproduce the problem and attach the log

[Alberto Milone (albertomilone)]

The problem with all-in-one systems that don't use connectors such as eDP is that we don't have a generic way to match the display with the input device. This means that we have to quirk such systems using the display EDID and the input controller.

[Alberto Milone (albertomilone)]

AceLan: please attach the output of the following commands for all the all-in-one systems you have available (with no external screens connected):

1) xrandr --verbose
2) xinput list --long
3) /var/log/Xorg.0.log

You can collect the data using a livecd of Ubuntu 14.04

[AceLan Kao (acelankao)]

Tested on Dell Inspiron 20 Model 3048, it works well.

[Yang Kun (YK) (dayangkuncn)]

Dell Inspiron 20 Model 3048 is an All-in-One system : ftp://ftp.dell.com/Manuals///all-products/esuprt_desktop/esuprt_inspiron_desktop/inspiron-20-3048-aio_Owner%27s%20Manual_en-us.pdf

[Alberto Milone (albertomilone)]

Yes, the Dell Inspiron 20 Model 3048 works without quirks because it uses an eDP connector.

I've just implemented a quirks system which should fix the Dell OptiPlex 9030 AIO and possibly other Dells with similar specs.

Please try the updated packages available here:
http://people.canonical.com/~amilone/lp1287341/

[AceLan Kao (acelankao)]

Here comes the Inspiron 2350 AIO log

[Alberto Milone (albertomilone)]

Ok, I've corrected the quirk for the OptiPlex and I've added one for the Inspiron 2350.

Please try the updated packages available here:
http://people.canonical.com/~amilone/lp1287341/

[AceLan Kao (acelankao)]

The latest unity-settings-daemon fix this issue on Dell OptiPlex 9030.
The touchscreen works well when connects an external monitor and it still works well after rotate the screen.

[AceLan Kao (acelankao)]

It works on Dell OptiPlex 9030 AIO

[Alberto Milone (albertomilone)]

Just an update: I've backported a system that should work better than single quirks. This is also what AceLan tested for me.

I've made a merge request into the bazaar branch of unity-settings-daemon, so that my code can be reviewed and accepted into 14.10. As soon as the merge is complete, we'll backport the feature to 14.04 (it's the same codebase).

[Launchpad Janitor (janitor)]

This bug was fixed in the package unity-settings-daemon - 14.04.0+14.10.20140604-0ubuntu1

---------------
unity-settings-daemon (14.04.0+14.10.20140604-0ubuntu1) utopic; urgency=medium

  [ Alberto Milone ]
  * gsd-xrandr-manager.c:
    - Add support for mapping the main touchscreen onto the laptop
      display (LP: #1287341).
      This makes sure that the input device knows exactly the area
      that represents the display when the screen configuration
      changes. Note: this doesn't cover the tablet use case.
    - Add support for matching displays with touch input devices
      according to the reported size. This is particularly
      useful on systems that don't use embedded display connectors
      i.e. all-in-one systems such as the Dell Optiplex 9030 AIO.
    - This work is a partial backport of the upstream work on
      touchscreens. When we finally sync with the upstream code
      we can drop this.
 -- Ubuntu daily release <email address hidden> Wed, 04 Jun 2014 15:42:47 +0000

[Colin Watson (cjwatson)]

Hello Steve, or anyone else affected,

Accepted unity-settings-daemon into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/unity-settings-daemon/14.04.0+14.04.20140606-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Ara Pulido (ara)]

Hello,

I was affected by this issue.

After installing unity-settings-daemon 14.04.0+14.04.20140606-0ubuntu1 from trusty-proposed I confirm that this is fixed.

I have tested that the touchscreen in my Dell XPS 13 behaves properly without an external monitor attached, and that it keeps behaving properly after connecting a non-touch external monitor to it.

I will mark this as verification-done

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for unity-settings-daemon has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package unity-settings-daemon - 14.04.0+14.04.20140606-0ubuntu1

---------------
unity-settings-daemon (14.04.0+14.04.20140606-0ubuntu1) trusty; urgency=medium

  [ Alberto Milone ]
  * gsd-xrandr-manager.c:
    - Add support for mapping the main touchscreen onto the laptop
      display (LP: #1287341).
      This makes sure that the input device knows exactly the area
      that represents the display when the screen configuration
      changes. Note: this doesn't cover the tablet use case.
    - Add support for matching displays with touch input devices
      according to the reported size. This is particularly
      useful on systems that don't use embedded display connectors
      i.e. all-in-one systems such as the Dell Optiplex 9030 AIO.
    - This work is a partial backport of the upstream work on
      touchscreens. When we finally sync with the upstream code
      we can drop this.
 -- Ubuntu daily release <email address hidden> Fri, 06 Jun 2014 14:39:15 +0000

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[dronus (paul-geisler)]

Still affects me on 14.04.

As it is already said, there is no reliable way to assign a touch device to a certain display device. So I suggest a two phase design:

1) Check if there is information for making an automatic assignment in some database. If there are more then one equal device, use display output and USB port id for identification.

2) If none is found, select the by some simple heuristics. Then launch a (to be made) settings program, that displays a centered button the first screen "PLEASE TOUCH HERE". If the button is pressed, the coordinates are checked against the center of all possible displays from 'xrandr' or gnome-display-settings information. If it matches, the assignment is stored to the local database. The procedure continues with the second screen, if there is a third one and so on.

If for some reason the database gets outdated (eg the user exchanges screens) phase 2 will be entered autmatically as you see.

The only caveat may be exchanging the output assignments of multiple equal display devices. For this case, the phase 2 program should be integrated into the system settings center, and easily be reached without proper touch operation (eg. launch via Unity by keyboard).

[Damiön la Bagh (kat-amsterdam)]

There is a regression of this bug in Ubuntu 20.04 using the default Gnome desktop

Setup is a Shuttle POS P511
using a Weida CoolTouch controller with driver integrated into the Linux kernel
Second screen is a AOC USB-3.1 Display Link monitor without display capabilities only (no touch).

When the AOC screen is disconnected the touch works normally.
When the AOC screen is connected the touch is out of sync and touches over both screens from the Shutlle/Weida screen.

Here is xinput list-props with both screens connected.

xinput list-props 11
Device 'Weida Hi-Tech CoolTouch® System':
 Device Enabled (169): 1
 Coordinate Transformation Matrix (171): 0.500000, 0.000000, 0.500000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
 libinput Calibration Matrix (308): 1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
 libinput Calibration Matrix Default (309): 1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
 libinput Send Events Modes Available (291): 1, 0
 libinput Send Events Mode Enabled (292): 0, 0
 libinput Send Events Mode Enabled Default (293): 0, 0
 Device Node (294): "/dev/input/event4"
 Device Product ID (295): 9589, 49920

[Damiön la Bagh (kat-amsterdam)]

Here is the xinput list-props while the second screen (AOC) is disconnected
Here the touch works as expected

xinput list-props 11
Device 'Weida Hi-Tech CoolTouch® System':
 Device Enabled (169): 1
 Coordinate Transformation Matrix (171): 0.500000, 0.000000, 0.500000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
 libinput Calibration Matrix (308): 1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
 libinput Calibration Matrix Default (309): 1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
 libinput Send Events Modes Available (291): 1, 0
 libinput Send Events Mode Enabled (292): 0, 0
 libinput Send Events Mode Enabled Default (293): 0, 0
 Device Node (294): "/dev/input/event4"
 Device Product ID (295): 9589, 49920

[Damiön la Bagh (kat-amsterdam)]

There is a workaround via

xinput map-to-output "Weida Hi-Tech CoolTouch® System" eDP-1

There needs to be some kind of detection to do this automatically and create a systemD systemctl file.