1 of 2
1
Introducing AS3 Multitouch Gesture Demo Application/Utility
Posted: 13 October 2009 02:39 PM   [ Ignore ]
Avatar
RankRankRankRankRankRank
Joined  2007-04-08
Total Posts:  2539
Dedicated

<center>?ACT=28&fid=33&aid=4490_wFFkkJVnBjvk7itSNeMo width=650</center>

On my last project I had to use a gesture to initiate an event. Due to this, I recently started experimenting in creating a simple multitouch version of some gesture engines that are available. I’ve come across three that are already available in AS3 and choose to start with the $1 gesture algorithm http://depts.washington.edu/aimgroup/proj/dollar/. It only seems fitting to start with this one since Andy Wilson of microsoft research (and many multitouch devices) had a hand in it’s development.

Like most algorithms, this one matches points drawn with a template that’s already stored. After the gesture is finished, it is matched up against templates in the database and given a percentage of how much it matched it’s closest template (from 0-1). One of the coolest things about this engine is that it’s orientation-less. This means, no matter how you’re orientated on the surface it’ll still recognize the gesture.

The main thing I contributed to the already created $1 gesture port was the ability to have multiple simultaneous gestures occurring at once. This means a single user can perform more than one gesture at a time or multiple users can perform their own gestures while someone else is using the device.

Currently, the demo isn’t fully segregated form the core gesture classes and therefore there’s still some work to be done so that the gesture classes can be used without the demo itself. In the meantime, go ahead and test this out and make sure to give some feedback; it also works with the mouse. The demo allows you to draw a gesture and if it matches on in the database, that gesture will ‘light up.’

The tolerance is set at 80% and therefore if a gesture doesn’t match 80% of a template, it’s not recognized.

DOWNLOAD

Image Attachments
gestureDemo.JPG
File Attachments
gesture.zip  (File Size: 103KB - Downloads: 1070)
 Signature 

MTmini, MTbiggie, & Audiotouch creator & Community Core Vision Co-founder

Follow on:
My Blog | Facebook | Twitter | Youtube

Profile
 
 
Posted: 13 October 2009 03:16 PM   [ Ignore ]   [ # 1 ]
RankRankRankRank
Joined  2009-08-21
Total Posts:  561
Elite

Wow man seems nice . Well using this we can have a gesture based menu system (like in PyMT) . Also i think we can make some games using this.  Also we can use this to have an editor which takes the gesture input and convert it into letters . one doubt though . I have no idea about the gesture class but was thinking is it possible to convert a combination of gesture to a particular gesture ? like when we write “A” people may have many combination to write (also many can write without lifting the pen) . So if there is possibility that a combination of gesture will work then it will be quite great .

Of course this application is great smile . i tried with mouse. got most of them right rasberry smile . keep up great work .

 Signature 

~~Amit~~

Website - Twitter - Facebook

Profile
 
 
Posted: 13 October 2009 07:59 PM   [ Ignore ]   [ # 2 ]
Avatar
RankRankRankRank
Joined  2006-11-09
Total Posts:  1499
Administrator

Really nice work seth looking forward to checking this out.. Is it possible to eidit the gesture library? Its sweet yo got multi gestures working too smile

Frm iphone

 Signature 

~

Profile
 
 
Posted: 14 October 2009 12:26 PM   [ Ignore ]   [ # 3 ]
Rank
Joined  2009-10-14
Total Posts:  6
New Member

Fantastic stuff. Yeah, really nice work, cheers for the link and source code.
Been wanting to tackle gestures for ages, so have spent the day working on it.
Got a basic handwriting multitouch gesture library(alphabet and numbers), based on this source code if anyones interested? feel free to email.

Thanks again for the source.

 Signature 

http://danfox.tumblr.com/

Profile
 
 
Posted: 14 October 2009 01:29 PM   [ Ignore ]   [ # 4 ]
Avatar
Rank
Joined  2007-07-13
Total Posts:  71
New Member

Well it is better to share the app here smile

Profile
 
 
Posted: 14 October 2009 02:13 PM   [ Ignore ]   [ # 5 ]
Avatar
RankRankRankRankRankRank
Joined  2007-04-08
Total Posts:  2539
Dedicated

Thanks guys.

Amit, it doesn’t currently allow for combined-multitouch gestures. The idea behind this isn’t really for handwriting or pinch/zoom type stuff as that’s not my personal focus or where I think gestures will be used in the future. It’s definitely possible to add such a feature and with the current library one could write: if a ^ followed by a - is done within 2 seconds, then make it an ‘A’.

It’s not implemented yet, but it’s possible to teach it more gestures and having an external xml file would be best for the future. A lot of the ones there are really too complicated to be reliable.

 Signature 

MTmini, MTbiggie, & Audiotouch creator & Community Core Vision Co-founder

Follow on:
My Blog | Facebook | Twitter | Youtube

Profile
 
 
Posted: 14 October 2009 07:36 PM   [ Ignore ]   [ # 6 ]
RankRankRankRank
Joined  2009-08-21
Total Posts:  561
Elite

Ahh I see Seth .Well if i get time i will surely study this to implement that idea . smile .Now I am a little busy . But it will be nice to have this. Is there any other way to edit or write in MT technology except having a keyboard (as it is same as having normal keyboard just on screen) . I hope defining all possible guesters will be different for all the letters . And by the way out of curiosity, how can i add an gesture template ? I mean do i really have to manually calculate all the points or is there some other method which does this ?

Sorry if i sound stupid as I never programmed for any gesture utility . rasberry

Regards
~~AMIT~~

 Signature 

~~Amit~~

Website - Twitter - Facebook

Profile
 
 
Posted: 14 October 2009 08:48 PM   [ Ignore ]   [ # 7 ]
Avatar
RankRankRankRankRankRank
Joined  2007-04-08
Total Posts:  2539
Dedicated

Right now I haven’t implemented a simple way to add templates, but it’s pretty easy. Basically you can add a trace statement and trace out the recognize array. It’ll print all the points in the format that the template takes. If you open the TouchGesture class you’ll see what I mean.

 Signature 

MTmini, MTbiggie, & Audiotouch creator & Community Core Vision Co-founder

Follow on:
My Blog | Facebook | Twitter | Youtube

Profile
 
 
Posted: 14 October 2009 08:52 PM   [ Ignore ]   [ # 8 ]
RankRankRankRank
Joined  2009-08-21
Total Posts:  561
Elite

Ahh got it . Thanks smile

 Signature 

~~Amit~~

Website - Twitter - Facebook

Profile
 
 
Posted: 18 May 2010 11:31 PM   [ Ignore ]   [ # 9 ]
Rank
Joined  2010-05-11
Total Posts:  9
New Member

@Cerupcat
How do I test out this gesture library, I ran CCV with TUIO enabled and then opened up the TouchGestureDemo.swf however I can only see a blank screen with the gesture pics and bottom right, do I need to install any other driver to get this up and running with Flash?
Thanks in advance.

Profile
 
 
Posted: 19 May 2010 01:28 AM   [ Ignore ]   [ # 10 ]
Avatar
RankRankRankRankRankRank
Joined  2007-04-08
Total Posts:  2539
Dedicated

Did you set the flash global security settings? Follow the directions here: http://nuigroup.com/forums/viewthread/1577/

 Signature 

MTmini, MTbiggie, & Audiotouch creator & Community Core Vision Co-founder

Follow on:
My Blog | Facebook | Twitter | Youtube

Profile
 
 
Posted: 19 May 2010 03:45 AM   [ Ignore ]   [ # 11 ]
Rank
Joined  2010-05-11
Total Posts:  9
New Member

I installed FLosc v2.0.4 and also enabled global security settings for flash on that folder however I still get a black screen when I run the gesture swf.
Is there anything I can do to debug the app, I have windows 7.
Is there any sequence to start up that is start flosc then ccv and then swf or any other way around.

Profile
 
 
Posted: 19 May 2010 04:51 AM   [ Ignore ]   [ # 12 ]
Rank
Joined  2010-05-11
Total Posts:  9
New Member

I even tried the “Getting started with TUIO and Flash” over here but even that doesn’t seem to be communicating any idea what could be going wrong.

Profile
 
 
Posted: 19 May 2010 01:18 PM   [ Ignore ]   [ # 13 ]
Avatar
RankRankRankRankRankRank
Joined  2007-04-08
Total Posts:  2539
Dedicated

If you’re using CCV, and sending to flash, you no longer need flosc. So make sure that isn’t running and try again.

 Signature 

MTmini, MTbiggie, & Audiotouch creator & Community Core Vision Co-founder

Follow on:
My Blog | Facebook | Twitter | Youtube

Profile
 
 
Posted: 19 May 2010 02:47 PM   [ Ignore ]   [ # 14 ]
Rank
Joined  2010-05-11
Total Posts:  9
New Member

Well I tried running only CCV and the swf but still no luck. How does CCV communicate to flash without FLosc, does it have FLOSC inbuilt ?
Also I don’t know where the error could be anything I could check?

Profile
 
 
Posted: 20 May 2010 12:36 AM   [ Ignore ]   [ # 15 ]
Rank
Joined  2010-05-11
Total Posts:  9
New Member

In CCV there are 3 settings TUIO udp, FLash XML and Binary TCP which one should I select. In fact I checked all three and no luck yet.
I also checked your source and there is a line in there
TUIO.init(this, ‘127.0.0.1’, 3000, ‘’, true);
where as in CCV the setting for TUIO udp is 127.0.0.1:3333 and for Flash XML and Binary TCP is 127.0.0.1:3000

Profile
 
 
   
1 of 2
1