[needs-packaging] reacTIVision - uses camera to track items on a table to generate Open Sound Control (OSC) commands
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Ubuntu |
Confirmed
|
Wishlist
|
Unassigned | ||
Bug Description
URL:
http://
Description:
reacTIVision is an open source, cross-platform computer vision framework for the fast and robust tracking of fiducial markers attached onto physical objects, as well as for multi-touch finger tracking. It was mainly designed as a toolkit for the rapid development of table-based tangible user interfaces (TUI) and multi-touch interactive surfaces.
reacTIVision is a standalone application, which sends OpenSound Control messages via UDP port 3333 to any connected client application. It implements the TUIO protocol, which was specially designed for transmitting the state of tangible objects and multi-touch events on a table surface. The framework includes a set of free example client projects for various programming languages, which serve as the basis for the development of tangible user interface applications. Alternatively reacTIVision is also able to send MIDI messages.
The reacTIVision application is available for the following operating systems: Windows, MacOS X and Linux.
Under Windows it supports any camera with a proper WDM driver, such as USB, USB2, FireWire and DV cameras. Equally under MacOS X all FireWire cameras and any camera supported by QuickTime will work in reacTIVision.
Under Linux, FireWire cameras are best supported, as well as a few Video4Linux and Video4Linux2 USB cameras.
How does reacTIVision work -
In a nutshell the system works like this: reacTIVision tracks specially designed fiducial markers in a real time video stream. The source image frame is first converted to a black&white image with an adaptive thresholding algorithm. Then this image is segmented into a tree of alternating black and white regions (region adjacency graph). This graph is then searched for unique left heavy depth sequences, which have been encoded into the fiducial symbol. Finally the found tree sequences are matched to a dictionary to retrieve an unique ID number. The fiducial design allows the efficient calculation of the marker's center point as well as its orientation. OSC messages implementing the TUIO protocol encode the fiducials' presence, location, orientation and identity and transmit this data to the client applications.
Additionally reacTIVision uses the result of the image segmentation in order to retrieve and identify small round white blobs as finger tips on the surface. A quick and dirty shape matching algorithm selects the actual finger blobs from the possible region candidates. A complementary blob tracking algorithm is also taking advantage of the same data in order to track eventually not recognized fiducials, for example where fast movements destroy the actual fiducial structure in the image.
The web site has instructions on how you can build your own table, or you can buy one.
There is a synthesizer and graphical component of the project, and this is not available for public use.
The Open Sound Control system is an improvement on Midi for controlling all sort of sound and multimedia devices, and other types of devices. see
http://
License: open source see http://
| Brian Murray (brian-murray) wrote | #1 |
*** This is an automated message ***
This bug is tagged needs-packaging which identifies it as a request for a new package in Ubuntu. As a part of the managing needs-packaging bug reports specification, https:/