Is anyone using or has anyone used Java for a multitouch app? Particularly using the open-table.org software?

not I… however I have used processing.org.. which is rather similar…

We’re using our own custom Java-based framework for blob tracking and gesture recognition. It is not freely available (sorry) due to several reasons, but might be at some point in the future. Anyway, if you’re having questions concerning Java & multitouch, I’m happy to answer them.

Thanks ElkMonster. I will likely have some questions and I hope you can help even though we will be using different platforms. Are you using a Java blob tracking platform only or are you also writing multitouch apps in Java?

The apps are also written in Java. The touch events/gesture events are sent as Java objects, so there would have to be a wrapper to use them with other languages.

I also am writing a multitouch framework for java which I will make available on sourceforge as soon as I complete it (some time in the near future, free time as the main variable).  I really found it a pain that there was no multitouch framework for java that I liked when I first got into multitouch, and so thought I might just break out of the box and make my own.  If anyone is interested in collaborating with me on this or the greater portion (the TUI aspect) of my project, or have suggestions, please respond with your email address and I will get in touch with you!

By the way, I’m not only willing to provide knowledge, but of course also code.
Not the framework as a whole (that would be too much confusing code anyway), but parts of it (depending on what you’re interested in), like for example camera capture code for Quicktime and JMF. I’m currently trying to capture from a Firefly under Linux using libdc1394 thought JNI. I’ll provide the code when/if it works.

ElkMonster - 30 June 2008 07:41 AM

By the way, I’m not only willing to provide knowledge, but of course also code.
Not the framework as a whole (that would be too much confusing code anyway), but parts of it (depending on what you’re interested in), like for example camera capture code for Quicktime and JMF. I’m currently trying to capture from a Firefly under Linux using libdc1394 thought JNI. I’ll provide the code when/if it works.

Great!  I’d love to see what you’ve been working on!

Woohoo! :D
I cannot believe it myself, but I got direct capturing with libdc1394 and JNI working under Linux (the latest Ubuntu). It wasn’t that hard actually, the code was mainly taken from some PtGrey example. I didn’t do anything with JNI before though.

The great thing about it is that no JMF needs to be used (I was kind of scared of it).

I’ll release all the code of it as soon as I find the time. Currently, my time is very limited because this all belongs to my bachelor thesis, and the deadline for everything to work (hardware & software) is in 10 days… so please be patient.

Great! I’m glad to see that some people are using Java. I didn’t want to be the only one because then I would have thought that everyone else knew something that I didn’t and I would find it out the hard way.

I can’t believe that everyone is working on their own blob tracking software though. I even considered it myself a few months ago. I wouldn’t mind seeing bits of your code ElkMonster if it will help me figure out how to get started writing applications. I’m used to looking at complicated code.

Legsmacgee, if I find Open Table to be a lackluster framework then maybe I can collaborate with you on yours. Ultimately though I would rather focus on development of multitouch applications. As long as the blob tracking software is fast and not overly difficult I would be willing to work with it.

I hope you both stick around on this site so we can help each other along. Apparently ElkMonster is furthest already.

Alpha Algorist - 01 July 2008 01:41 AM

I can’t believe that everyone is working on their own blob tracking software though. I even considered it myself a few months ago.

Well, the reason in my case is that when we (two friends and me back then) started with the project, there was no Touchlib yet, and we needed something that worked on OS X. During that semester, Touchlib appeared, but it was obviously sluggish (and OS X compatibility uncertain (and I thought their filter concept was much too complicated)). We had the basics of our framework done, so I kept working on and with it. Also I didn’t need to take into account other’s opinions - the good thing about your own framework is that you can mess with it in every way you like. That’s also why it has become complex and untidy…

I wouldn’t mind seeing bits of your code ElkMonster if it will help me figure out how to get started writing applications. I’m used to looking at complicated code.

I hope you both stick around on this site so we can help each other along. Apparently ElkMonster is furthest already.

Well, it may be possible to open all the code to the public… The main issue is a legal one - the code is part of a university project and the mentoring professor is sceptic about releasing the code to the public. I had the idea in my mind from the beginning, and I actually asked the professor already one year ago about making the framework open source… well, I can and will ask again, but if he says no, then I can’t do anything about it. :(

Anyway, as stated above, I’m short of time currently, so please be patient.
However, if you would like to see or know how certain things are done, simply name them and I’ll see what I can do for you currently.

I am one of those, who worked together with ElkMonster on this Java implementation. I am not a project member any more, but I can tell, that there is no magic in the code. I am quite sure, that we can publish the core components, like the blob-recognition, the blob-tracing and the filter-chain, as this is our intellectual property – as long as we do not publish the applications we made/make for the university on top of that. But as ElkMonster I don’t have time to make an library for distribution right now.

vikingosegundo - 01 July 2008 07:51 AM

I am one of those, who worked together with ElkMonster on this Java implementation. I am not a project member any more, but I can tell, that there is no magic in the code. I am quite sure, that we can publish the core components, like the blob-recognition, the blob-tracing and the filter-chain, as this is our intellectual property – as long as we do not publish the applications we made/make for the university on top of that. But as ElkMonster I don’t have time to make an library for distribution right now.

I work on a research project (TM4L) at my university and despite several companies who have requested it, my research professor won’t open source our project either. If the framework was created as part of the research your professor may not allow you to release it even if you wrote it. I think the Open Table software will be great because it will allow me to use swing to build multitouch apps. That will also benefit many other students who will be using my table because our first two programming courses use Java exclusively.

I have some questions for you both. Please take your time in responding. This is definitely not urgent information on my end. What kind of applications did you guys create? What did you use to create your application interfaces? JOGL or something similar? Also, I noticed that you said that your code did gesture recognition in a previous post. Do you guys have predefined gestures to do things like single and double clicks? What other kinds of gestures does your software recognize? I think I would be most interested in that because I would like my application to have gesture recognition to open certain menus to save screen space. If Open Table doesn’t do this already I will likely want to modify it.

Finally I have some time to answer your questions.
Sorry it took so long.

What kind of applications did you guys create?

The first thing we created was an application that was part of a media space. It let people in different locations share documents through drag-and-drop. Then followed two implementations of pong-alike games, the second of which is now part of a study for my bachelor thesis. Furthermore there exist a number of simple test applications for mouse emulation (only single-touch though), primitive painting, and weird midi sound making. :D

What did you use to create your application interfaces? JOGL or something similar?

The media space application and the tests are using AWT/Swing, the games are made using the wonderful Slick (which uses LWJGL - and thus JOGL, I think).

Do you guys have predefined gestures to do things like single and double clicks? What other kinds of gestures does your software recognize?

The gesture recognition built into the framework is rather the basic infrastructure than a set of predefined gestures. You can define the gesture set at runtime, so it is possible to change the kinds of gestures without the need to restart the programme.
Gestures are defined by a decision tree. Each single-touch event (i.e., blob/path) is going through the tree from the top. The leaves of the tree are the output nodes, that is, the nodes that create multi-touch events.

“Multi-touch event” and “gesture” are somehow synonyms here; you could say that the outcome of a gesture is a multi-touch event that is of the type of the recognised gesture. These events are then sent to the application which performs the according actions. Multi-touch events are not necessarily generated for each frame, but it depends on the gesture. For example, one could implement a group select gesture which lets the user draw a circle around some objects to select them. The according event would only be fired after the user has completed the circle, but not in between.

We have an example implementation of the basic gestures (press, release, move/scale/rotate), however it depends an the application whether/to what extend they can be used. Also, they need quite some rework. :D

Concerning double click gestures, I think it would be possible to implement that relatively easily - the details might get nasty though. But that’s true for the whole gesture recognition thing… at first it seems quite straightforward, but one gets confronted with the fiddly details pretty soon. Some of these details are: What happens when the object under a blob changes between two frames? How do you prevent gestures from being incompatible with one another (or rather, too compatible/very similar)? What do you do when a gesture is performed, but another blob enters the scenery that alters the context?

My basic approach for parts of these problems is that blobs on the same object belong to one multi-touch event, and any changes in the number of blobs on an object require a full re-evaluation from the beginning.

Well, I think I’m getting lost in the details now, while at the same time I’m not sure which information is relevant and which isn’t.
So, maybe it’s better for you to just keep asking instead of me talking weird.

Sweet!  Mine is finally done too.  Its still in its infancy, but it works pretty well.  I’ve just been using it for some standard drawing/photo manipulation apps, but plan to move into some cooler stuff soon.

hi ElkMonster, i’m working on a JNI camara capture (firefly) via libdc implementation (os-x + processing.org user) myself. well, “working” might not be the right term, “planning on” is. since i’m not into reinventing the wheel too much, would you consider sharing that part of your code?

thanks a ton
F