On my last project I had to use a gesture to initiate an event. Due to this, I recently started experimenting in creating a simple multitouch version of some gesture engines that are available. I’ve come across three that are already available in AS3 and choose to start with the $1 gesture algorithm http://depts.washington.edu/aimgroup/proj/dollar/. It only seems fitting to start with this one since Andy Wilson of microsoft research (and many multitouch devices) had a hand in it’s development.

Like most algorithms, this one matches points drawn with a template that’s already stored. After the gesture is finished, it is matched up against templates in the database and given a percentage of how much it matched it’s closest template (from 0-1). One of the coolest things about this engine is that it’s orientation-less. This means, no matter how you’re orientated on the surface it’ll still recognize the gesture.

The main thing I contributed to the already created $1 gesture port was the ability to have multiple simultaneous gestures occurring at once. This means a single user can perform more than one gesture at a time or multiple users can perform their own gestures while someone else is using the device.

Currently, the demo isn’t fully segregated form the core gesture classes and therefore there’s still some work to be done so that the gesture classes can be used without the demo itself. In the meantime, go ahead and test this out and make sure to give some feedback; it also works with the mouse. The demo allows you to draw a gesture and if it matches on in the database, that gesture will ‘light up.’

The tolerance is set at 80% and therefore if a gesture doesn’t match 80% of a template, it’s not recognized.

DOWNLOAD

Wow man seems nice . Well using this we can have a gesture based menu system (like in PyMT) . Also i think we can make some games using this.  Also we can use this to have an editor which takes the gesture input and convert it into letters . one doubt though . I have no idea about the gesture class but was thinking is it possible to convert a combination of gesture to a particular gesture ? like when we write “A” people may have many combination to write (also many can write without lifting the pen) . So if there is possibility that a combination of gesture will work then it will be quite great .

Of course this application is great . i tried with mouse. got most of them right . keep up great work .

Really nice work seth looking forward to checking this out.. Is it possible to eidit the gesture library? Its sweet yo got multi gestures working too

Frm iphone

Fantastic stuff. Yeah, really nice work, cheers for the link and source code.
Been wanting to tackle gestures for ages, so have spent the day working on it.
Got a basic handwriting multitouch gesture library(alphabet and numbers), based on this source code if anyones interested? feel free to email.

Thanks again for the source.

Well it is better to share the app here

Thanks guys.

Amit, it doesn’t currently allow for combined-multitouch gestures. The idea behind this isn’t really for handwriting or pinch/zoom type stuff as that’s not my personal focus or where I think gestures will be used in the future. It’s definitely possible to add such a feature and with the current library one could write: if a ^ followed by a - is done within 2 seconds, then make it an ‘A’.

It’s not implemented yet, but it’s possible to teach it more gestures and having an external xml file would be best for the future. A lot of the ones there are really too complicated to be reliable.

Ahh I see Seth .Well if i get time i will surely study this to implement that idea . .Now I am a little busy . But it will be nice to have this. Is there any other way to edit or write in MT technology except having a keyboard (as it is same as having normal keyboard just on screen) . I hope defining all possible guesters will be different for all the letters . And by the way out of curiosity, how can i add an gesture template ? I mean do i really have to manually calculate all the points or is there some other method which does this ?

Sorry if i sound stupid as I never programmed for any gesture utility .

Regards
~~AMIT~~

Right now I haven’t implemented a simple way to add templates, but it’s pretty easy. Basically you can add a trace statement and trace out the recognize array. It’ll print all the points in the format that the template takes. If you open the TouchGesture class you’ll see what I mean.

Ahh got it . Thanks

@Cerupcat
How do I test out this gesture library, I ran CCV with TUIO enabled and then opened up the TouchGestureDemo.swf however I can only see a blank screen with the gesture pics and bottom right, do I need to install any other driver to get this up and running with Flash?
Thanks in advance.

Did you set the flash global security settings? Follow the directions here: http://nuigroup.com/forums/viewthread/1577/

I installed FLosc v2.0.4 and also enabled global security settings for flash on that folder however I still get a black screen when I run the gesture swf.
Is there anything I can do to debug the app, I have windows 7.
Is there any sequence to start up that is start flosc then ccv and then swf or any other way around.

I even tried the “Getting started with TUIO and Flash” over here but even that doesn’t seem to be communicating any idea what could be going wrong.

If you’re using CCV, and sending to flash, you no longer need flosc. So make sure that isn’t running and try again.

Well I tried running only CCV and the swf but still no luck. How does CCV communicate to flash without FLosc, does it have FLOSC inbuilt ?
Also I don’t know where the error could be anything I could check?

In CCV there are 3 settings TUIO udp, FLash XML and Binary TCP which one should I select. In fact I checked all three and no luck yet.
I also checked your source and there is a line in there
TUIO.init(this, ‘127.0.0.1’, 3000, ‘’, true);
where as in CCV the setting for TUIO udp is 127.0.0.1:3333 and for Flash XML and Binary TCP is 127.0.0.1:3000