NUI Group » Wiki

TUIO

2008-08-20T11:50:28-05:00

TUIO is a simple yet versatile protocol designed specifically to meet the requirements of table-top tangible user interfaces. This protocol defines common properties of controller objects on the table surface as well as of finger gestures performed by the user. Initially the protocol has been implemented within reacTIVision, the computer vision engine developed for the reacTable project. It also has been adopted by several other projects related to tangible and multi-touch interaction, such as the touchlib and the NUI group community. The TUIO protocol is based on OpenSound Control and is therefore usable on any platform supporting this protocol. There are many example clients available for programming languages such as C/C++, C#, Java, Objective C, Python, Smalltalk, Actionscript and multimedia environments such as Pure Data, Max/MSP, Quartz Composer, SuperCollider and Flash.

TUIO was designed by Martin Kaltenbrunner, Ross Bencina, Enrico Costanza and Till Bovermann during a short-term scientific mission supported by the COST287-ConGAS project.

TUIO protocol 1.0 specification: http://reactable.iua.upf.edu/?tuio
official TUIO example clients (part of reacTIVision) http://reactable.iua.upf.edu/?software

TUIO sources:

http://reactivision.sourceforge.net/ reacTIVision:
the reacTable computer vision framework with a complete TUIO implementation
http://www.nuigroup.com/touchlib/ Touchlib:
an application framework for multi-touch surfaces based on FTIR and DI
http://code.google.com/p/wiimotetuio/ WiimoteTUIO:
an application that sends the locations of all IR sources found
http://lux.vu/blog/oscemote/ OSCemote:
an iPhone remote control that includes a TUIO multi-touch controller

For more info on programming languages and environments that have TUIO support, read Writing TUIO Apps

Category:Software

community cams

2008-08-07T01:50:49-05:00

Fill in the camera you are using. This to give an overview of the different cameras the people use inside this community. Please edit in alphabetical order by manufacturer and then model.

*DanKn - Creme de la Crap, no name brand. Frame rate:640 x 480 30 f/s;1280 x 960 10 -15 f/s USB 2.0
*jason - Logitech Quickcam Chat (budget), IR filter removed, 7 fps @352x188 on Linux, USB 1.1
*alan - Logitech Quickcam Communicate STX, removed IR Filter, 30fps @ 640x480.
*BBatchelder - Microsoft VX-6000, IR filter removed, 30fps @640x480, USB 2.0.
*Bilsky - Phillips SPC900-NC
*Chrizz - Phillips SPC900-NC (mounted a Philips PCVC740K lens, removed ir-filter)
*Davman - Phillips SPC900-NC
*paulzain - Point Grey Dragonfly2 + Computar 1.8mm lens
*Pleh - Phillips SPC900-NC
*Sidje - Samsung VP-D101 FW with build-in but bad IR filter - Creative webcam 3 USB1.1
*Recklessness - Sunplus CA533A 320x240 @ 30FPS, no IR, experimenting with ambient diffuse light.
*Gravano - Sweex WC001 (budget) Phillips SPC900-NC (for improved performance)
*jibberia - Point Grey Firefly MV
*thomas.hansen - Point Grey Firefly MV
*juwaidah1990 - Logitech QuickCam Messenger, IR filter removed, Linux compatible.
*BlaXwan - Unibrain Fire-i board camera, firewire, you can order it with lens without ir-filter (4.3 and 1.9 mm fish-eye lens), 640x480@30fps
*Matthew180 - Sony MiniDV DCR-HC40, firewire, 720 x 480 @ 60f/s. Has a “Night Shot” mode which internally removes the IR filter so it is perfect for capturing the IR blobs. I had this camera already so I used it, but probably overkill and certainly an expensive way to go.
*Blue - Intel YC76 with negatives as IR filter
*Robin - Trust Spacecam 360 with black and white photo negative as IR filter. Runs at 30fps 640x480.
*zviangi - Unibrain Fire-i B/W camera with 2.10mm wide angle lens, no IR coating
*skeeet - microsoft VX-3000 640x480@30fps, Xbox360 cam 320x240@60fps, both with negative film as filter

Quartz Composer with BBTouch

2008-08-05T13:30:38-05:00

This article will show you how to get a simple single touch cursor functionality in Quartz Composer using BBTouch. This only works on Leopard.

1) Get and install Quartz Composer and Xcode

At the moment I assume you already have quartz composer, it’s included with the developer DVD and available as a free download from apple. Check here for more info.

2) Get and install TUIO plugin
Download the Tuio_Quartz package from the reacTIVision software page:

Unpack the archive and copy TuioClient.plugin to /Library/Graphics/Quartz Composer Plug-Ins

If you have already put a patch called TuioClient.plugin and put it in /Library/Graphics/Patches, delete it.

3) Get and install BBTouch from the googlecode project.

Download the latest version in the googlecode list, or compile it yourself from source. If you download the source, don’t worry about anything except BBTouch.

4) Load and go!

Open BBtouch, configure it so that you are getting nice consistent blobs in the right window. If you are using a MTMini then make sure the box called “Find Dark Blob’s is checked.

Open Quartz Composer and make a new blank composition.

From your patch creator, first put in a TuioClient. Then drop in a Structure Index Member. Connect the “cursors” of the TuioClient to “structure” of the Structure Index Member.

Find a Structure Key Member and drop 2 of those in your comp, one for the x-position and one for the y-position.

In one of the Structure Key Member patches, for the Input Paramaters type in xpos.

In the other Structure Key Member patch, for the Input Paramaters type in ypos.

Connect “member” from the Structure Index Member to “structure” of each Structure Key Member.

Now load a renderer, the teapot seems to be popular put a picture attached to a billboard works great too. Let’s just go with teapot. Drop the teapot patch in.

Change the teapot scale to .25. Attach the “member” from your xpos Structure Key Member patch to the “X Position” of the teapot. Do the same thing for your ypos, this time attaching to the “Y Position” of the teapot.

Assuming you have everything working, that should be it! The teapot will follow your blob, although it might need to be calibrated.

Writing TUIO apps

2008-08-05T13:26:29-05:00

There are a number of programming languages and multimedia environments that support the TUIO
This is a list of example clients and libraries that can be used for developing multitouch applications:

Programming Languages:
C/C++ reacTIVision framework
Java reacTIVision framework
C# reacTIVision framework
Objective C
Python
Squeak (SmallTalk)
ActionScript (see the AS3 directory in the touchlib source)

Multimedia Environments:
Pure Data reacTIVision framework
Max/MSP reacTIVision framework
Quartz Composer reacTIVision framework
VVVV
SuperCollider
Flash reacTIVision framework

quartz composer

2008-07-30T01:58:38-05:00

Quartz Composer is included with the Apple Developer tools disk, also from their website.

OpenSound Control

2008-07-29T20:14:05-05:00

OSC or OpenSound Control is a protocol for the communication between controllers and sound synthesizers, but has been also adopted for other applications or multimedia devices. OSC was primarily designed as a replacement for MIDI in order to overcome the bandwidh and speed limitations of this standard protocol for digital musical instruments.
Although OSC is not directly bound to any particular transport method, most implementations are based on the UDP network protocol, using standard internet/network ethernet connections. Therefore it can be run it on a single machine (using the localhost), between two networked machines, or over the internet in general.

The TUIO protocol, which was specially designed for transmitting the state of tangible objects and multi-touch control on a table surface is based on OpenSound Control, and is used by Touchlib and reacTIVision for the communication with the actual applications.

Be sure to check out Writing TUIO apps

Category:Software

Building Your First Application

2008-07-27T19:49:45-05:00

1: create a new Flash file (actionScript3)
2: click on the canvas, the properties for the file should not be available.
3:click on the size button.
4: change that to 1024 x 768 and 30+ FPS
5: click settings (below size button)
6: Where it says actionScrtip version, make sure ActionScript is selected and click settings
7: in classpath at the bottom, click on the + button and enter
../ext
do the same for
../int
8: save this flash file in (the touchlib folder) AS3/src (name it what you want) EG touch.fla
9: in the AS3/int/app/demo folder, create a new folder called MyTouchApp (must be exact)

10: create a new ActionScript file.
11: save it as MyTouchApp.as in AS3/int/app/demo/MyTouchApp (that folder you just made)
12: go back to (click on the tab for) you flash file EG. touch.fla

13: In the Document Class text field enter app.demo.MyTouchApp.MyTouchApp

14: select the rectangle button and make a rectangle that is over, and larger than your canvas. This will be the area your program will run on.
15: right click (control click on a mac) on the black rectangle you just made and select “Convert To Symbol”
16: make sure movieclip is selected and press ok.
17: Save file… go back to MyTouchApp.as
18: copy code from examples and paste it in this file. MAKE SURE THE PAGE IS CLEARED FIRST
19: save this file again, go back to (click on the tab for) you flash file EG. touch.fla

20: publish the file. It should work now

Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test

Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test

==Code Examples==

package app.demo.MyTouchApp{ //adds a circle where you touch (no resize)

    import flash.display.*;        
    import flash.events.*;
    import flash.net.*;
    import flash.geom.*;        

    public class MyTouchApp extends Sprite {

        public function MyTouchApp() {

            //--------connect to TUIO-----------------
            TUIO.init(this,'localhost',3000,'',true);
            trace("MyTouchApp Initialized");
            //----------------------------------------        

            addEventListener(TouchEvent.MOUSE_DOWN, touchDown); //run touchdown, when touched
        }
        
        public function touchDown(e:TouchEvent):void{        
                        
            var curPt:Point = parent.globalToLocal(new Point(e.stageX, e.stageY)); //convert touch points to x,y                
            
            var circle:Sprite = new Sprite(); //create a new sprite
            
            circle.graphics.lineStyle(10, 0xff0000); //set line width to 10px and red
            circle.graphics.drawCircle(0,0,40); // draw a 40px circle
            circle.x = curPt.x; //put it where touch is (x cord)
            circle.y = curPt.y; //put it where touch is (y cord)

            addChild(circle); //add the circle where touch happened
            
        }
    }
}

package app.demo.MyTouchApp{

    import flash.display.*;
    import flash.events.*;
    import app.core.action.RotatableScalable;

    public class MyTouchApp extends RotatableScalable {

        public function MyTouchApp() {

            //--------connect to TUIO-----------------
            TUIO.init(this,'localhost',3000,'',true);
            trace("MyTouchApp Initialized");
            //----------------------------------------        

            addEventListener(MouseEvent.MOUSE_DOWN, onMouseDown); // adds an event listener looking for a mouse click - runs "onMouseDown"
        }
        
        
        private function onMouseDown(event:MouseEvent):void { //creats a circle when the mouse is clicked

            var circle:Sprite = new Sprite();
            
            circle.graphics.lineStyle(10, 0xff0000); //set line width to 10px and red
            circle.graphics.drawCircle(0,0,40); // draw a 40px circle
            circle.x = mouseX; //put it where the mouse clicked
            circle.y = mouseY;

            addChild(circle); //add the circle to the plane
        }
    }
}

index

2008-07-27T04:22:22-05:00

Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications. Read more...

This wiki contains information about the software & hardware related to multitouch and multi-model input systems. Also you will be able to find out general information about our community here.

Featured Wiki Articles

Building Process
Cameras
Touchlib

Community Links

Home
About Us
News Archive
Forums
IRC - #nuigroup on irc.freenode.net

Community Projects

OpenTouch
Lux Platform
Touchlib
TouchEvent
GSoC

Wiki Help

Note: Wiki theme broken in IE… get a real browser…

AS3

2008-07-23T00:13:03-05:00

#REDIRECTActionscript 3

Multitouchable

2008-07-23T00:12:13-05:00

To use this class in AS3:

import app.core.action.Multitouchable;