MultiTouch compatibility and features

[Daniel Schürmann]

Our fellow digital DJ software designers (our competitors!) have already started to develop and release products for touch and multitouch interfaces. This control interface has fast become ubiquitous in the computing world and is not only here to stay, but will spread and transform into other control systems, probably largely gesture based. All of these touch control interfaces can be guaranteed to have one detail in common: MultiTouch.

Touchscreen hardware of any kind currently seems scarce among the development team. Therefore it will be valuable for anyone in possession of such hardware to install Mixxx on it and begin testing and reporting the experience and issues as new bugs here on Launchpad.

Mixxx needs to catch up in this particular area and eventually innovate. Current research suggests that implementing this: https://qt-project.org/doc/qt-4.8/qtouchevent.html aspect of the QT framework may be the place where we need to begin in order to implement multitouch. Currently, "singletouch" works mostly fine with the exceptions of Bug #1217733 and Bug #1217738, both of which might possibly affect multitouch as well.

Lastly, it is worth noting here that problems of "the buttons are too small for my fingers!" can very likely be solved easily through skin design. Ghutalyn, for instance, features many larger buttons that seem to fit well with the finger-based control of touch application.

Do not report individual touch-related bugs here at this Bug (#1152572). This bug is exclusively and generally regarding the lack of MultiTouch support - specific and individual singletouch and multitouch-related problems or feature requests should be reported individually as new bugs.

[jus (jus)]

What you mean exactly by "starting to release solutions for touch interface"?
iOS releases like Traktor DJ or CrossDJ?

The only touch interface we could possibly test now is a SurfacePro running Win8.

[Daniel Schürmann (daschuer)]

I filed this bug after reading this German article:
http://www.maclife.de/beat/news/traktor-dj-ni-bringt-den-klassiker-auf-s-tablet

Qt is able to handle touch events:
http://qt-project.org/doc/qt-4.8/qtouchevent.html

So we need at least one developer with access to a tablet pc.

[jus (jus)]

Hm, ok all cool and I would love see running Mixxx on tablet device too, but we are talking about porting to iOS or Android, right?
To my knowledge QT is only available as preview for iOS/Android: http://blog.qt.digia.com/blog/2013/03/05/qt-for-ios-preview/

Access to iPad/Nexus device would be not the issue.

[Daniel Schürmann (daschuer)]

Porting Mixxx to additional platforms is a separate and challenging issue.

I have focused touch setups based on one of our current supported platforms when writing this bug.

[RAWRR (rawrr)]

jus: "What you mean exactly by "starting to release solutions for touch interface"?
iOS releases like Traktor DJ or CrossDJ?"

Also CuePlayDJ for iOS. Haven't used it in person, but watched it evolve over multiple versions in multiple YouTube videos - it seems to be extremely sophisticated at this point.

[RAWRR (rawrr)]

jus: "The only touch interface we could possibly test now is a SurfacePro running Win8."

Not true, there have been touchscreens for Intel PCs for a decade now, in fairly broad distribution in various markets including the consumer market; they just haven't been multitouch.

Unless you mean that the SurfacePro is the only relevant testing platform *because* it is multitouch and *currently* on the market.

[Daniel Schürmann (daschuer)]

The issue of this bug is: You might use a touch screen with Mixxx right now. But there are for sure some reasons that prevents controlling Mixxx with touch only. So what are these issues and what is the right way to solve them?

http://qt-project.org/doc/qt-4.8/gestures-overview.html is a good starting point.

[RAWRR (rawrr)]

I have looked it over briefly already. It was exciting, thank you for that link. You know I'm a complete novice here, but from what I could read it looked like if Mixxx is compiled with a current version of QT, then it just means going over Mixxx's "code" (I barely know what I'm talking about when I say "code") and "tacking on" the relevant attributes to the various widgets and such, which almost certainly already have locations in the code. Am I right? I mean, it looks like some kind of a "search, paste, search, paste, search paste, search..." process, something you'd write a script for maybe.

I mean, I'm suggesting that it looks like there is no outright mystery - just digging and pasting followed by tons of testing.

For me it would be hard because I'd have to learn thousands of control locations... not expeditible.

[Daniel Schürmann (daschuer)]

Forum reference
http://www.mixxx.org/forums/viewtopic.php?f=1&t=5338

Hi RAWRR,

Do you have a touch device that is able to run Mixxx?
If yes, it would be nice if you can try to control Mixxx with touch only. If you find issues, it wold be nice if you could file separate bugs for them + an Idea how this feature might be controlled in a touch enabled version of Mixxx.

The fix itself will not be much of copy and past, because the control engine is well designed without much code duplications.
Are you able to build Mixxx from source?

[RAWRR (rawrr)]

Attached to this post is a video of simple operation with an older monotouch touchscreen.

Specs are:

Mixxx version - 1.11.1-pre (r3881)
computer: Panasonic cf-73
cpu - 1.7ghz Pentium M - D:
spu - onboard
ram - 1.5gb
OS - WinXPSP3
touchscreen (built in) - Fujitsu

This touchscreen has finger (and glove, for outside use) capability, but it works best with a stylus of some sort. You can see in the video the lag with the fingers - it has nothing to do with Mixxx, every other interface in the system lags by finger as well.

The only issues Mixxx has with a **monotouch/singletouch** interface that I have found are as follows:

Right-clicking, according to my driver features, happens by assigned keyboard key+touch. In other words, if you want to right-click you assign a keyboard key (in my case one of three) and whatever you touch while that key is pressed becomes a right-click. You can also set it to be pressed and let go once, after which the next thing you touch becomes right-clicked (useful for monotouch screens without separate keyboards).

In the case of Mixx4Touch, accommodating this could be as simple as having a button in the interface assignable to one of these keys. I imagine that other PC touchscreens function in a similar way, especially since a great many of them are Fujitsu. To accommodate OSs such as Android or others built especially for tablets without mice or keyboards, Mixxx would have to come up with an alternative to right-click for some features. Come to think of it, here's a probably obvious thought: how does Mixxx for Mac get by without right click? Oh, you Mac people and your frightening one-click mouses.

In any case, any OS meant to be used with a right+left click mouse really doesn't need that much accommodation, other than adding the QT "touch" controls for multitouch.

The second and far more crippling problem is that the dial controls are wonky as *&&*^. I tried to show that in the video - the cursor of course, as usual, raises and lowers them smoothly, but for some reason the touch at the dials is extremely sensitive, or just wrong. I can't make it out where it goes wrong or why. All sliders, buttons, menus, and every other clickable item, as far as I can tell, work perfectly.

Anyway, that really is it as far as barebones functionality on a monotouch screen. Multitouch will be another matter, I think the touch-specific controls debuted in tandem with multitouch functionality. The cf-73 and many other tablets were manufactured long before Windows 7 or Android were even invented. They did not need any special libraries, just their driver, like any other hardware. Older touchscreens were made like mice, one clickpoint and made to work with any kind of legacy program. As far as I can tell multitouch was basically what exploded the paradigm and forced programs to be designed especially for it.

I'll be on the lookout for a cheap multitouch tablet or laptop, or possibly a Wacom Bamboo will work (which is also multitouch, and probably the cheapest option) in order to test multitouch functionality better. Actually, never mind - I can totally just build the MTMini whenever QTtouch gets added. I should be able to build it...

[RAWRR (rawrr)]

@ daschuer -

To answer your question: I do not know how to build Mixxx from source currently. When Mixxx began migrating to GitHub I resolved to try, and so acquired my first IDEs a week or two ago. I haven't gotten the chance to work with them yet.

[Daniel Schürmann (daschuer)]

Thank you very much for your testing and your video.

So we have two main Issues:
1. We need a way to right click pushbuttons with a single touch device Bug #1217733
2. Rotaries are hard to control with single touch device Bug #1217738

I am currently working on the pushbutton so i will take care for 1.

Here is a nice block How win8 solves the issue.
http://techbymak.com/how-to-right-click-on-touch-screen-laptop-tablets-windows8.html

[RAWRR (rawrr)]

jus: "The only touch interface we could possibly test now is a SurfacePro running Win8."

Here is an addendum to "other-than-SurfacePro" Multitouch-enabled devices currently in production and running a Mixxx-compatible OS:

-Acer Aspire R7
-ASUS Transformer Book
-MSI Slidebook s20
-ASUS Taichi
-Toshiba Satellite U925t
-Lenovo Ideapad Yoga
-Dell XPS (10+12)
-HP Envy x2
-ASUS ViviTab
-Lenovo ThinkPad Helix
-Toshiba KIRAbook
-Sony VAIO Pro 13
-ASUS VivoBook S550
-ASUS VivoTab Smart
-Dell Latitude 10 Tablet
-Acer Iconia W700
-Acer Iconia W3-810

I compiled this list off the cuff; I may have accidentally included one or two devices shipped with RT, which I'm sure we all hope meets a quick demise.

I won't keep updating these lists or anything. This is just to provide an example of the presence and prevalence of x86/x64 processor systems with multitouch screens.

[Daniel Schürmann (daschuer)]

This explains how touch events are handled in Linux:

http://who-t.blogspot.de/2010/10/thoughts-on-linux-multitouch.html
http://who-t.blogspot.de/2011/12/multitouch-in-x-pointer-emulation.html

[Swiftb0y (swiftb0y)]

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/6939