Here’s an interim approach… Since TUIO has a lot of momentum and it will take a some reshuffling of code to move to whatever is next (TUIO2 or LusidOSC), it could be helpful to have a gateway that takes in one protocol and restructures the data (as well as possible) for another protocol.

There is now a LusidOSC / TUIO Gateway that does this (see the bottom of http://lusidosc.sourceforge.net/ ) built in processing so that it can easily be modified.  It could use a little more work (right now it only handles one profile at a time), but it is a nice idea for a middle ground as the community prepares for how to move forward.

Adam

Hi adam,

I’m glad you could join the conversation. I think there’s a lot of great points you and Martin made here and I hope we can openly discuss both TUIO and lusidosc since this really could potentially have a big impact on the future no matter what direction things lead.

Could you explain more about this ‘gateway’ between LusidOSC and TUIO? Would there be a possibility for tuio to lusidosc also or is the protocol not reversible in this way.

Hi Seth,

Thanks for the welcome!  I have a lot of respect for the great ideas NUI Group is pursuing.

With regards to the gateway, the protocol should be reversible as well; allowing for TUIO messages to be rearranged as LusidOSC messages.  The reason I started with LusidOSC to TUIO is simply because the number of applications is far greater than the number of different sensing systems.  The gateway should allow us to keep older applications working until they can be updated to the newer protocol. 

If there is interest in making a gateway in the other direction, we can definitely pursue it.  The source is available and runs in Processing, so with just a few changes it could listen for TUIO messages and output LusidOSC. 

I’m looking forward to continuing the discussion of how to shape the protocol to integrate with as many of these new and exciting interfaces as possible.

Hi Adam

Have you already had a look at the Tuio2 spec draft? I think it’s a great step on being more kindle to other forthcoming devices. more versatile and flexible.

Thanks for the link to the TUIO2 spec draft!  It looks like there has been a lot of thought put into it, along with some good improvements.

However, I think that TUIO2 is a temporary fix for a much larger problem.  The problem is two-fold: 1.) a few months from now, when NUI Group finds the next important kind of data to send along, we will have to start the process all over again (hacking some changes together, rewriting libraries, and requesting an updated version of TUIO to support our ideas, if we’re lucky), and 2.) TUIO is Reactable-centric (understandably, for their purposes) and is not designed to support the very large range of new input devices this community, and many others, are pushing forward in new and exciting ways. 

LusidOSC is an open initiative to address both of those problems.  It’s built on top of the ideas of TUIO, but with the community support, scalability, and stability we need to continue exploring crazy new interface ideas.  We owe much to the efforts of Martin Kaltenbrunner, Till Bovermann, Ross Bencina, and Enrico Costanza for their pioneering TUIO work; but I strongly believe that the protocol should rest in the hands of the communities using it.  We need a protocol that will scale across a LOT of different tracking systems so that developers can work together and build off of each others’ applications—that would be great for all of us! 

The second version of LusidOSC is already underway and everyone is encouraged to join in its development (see the “LusidOSC v1.1 document” on the LusidOSC website).  I hope NUI Group will give this new initiative serious consideration before making a decision about where to go next with regards to a communication protocol.

Adam

Hi Adam,
I’m sorry I have to strongly disagree with your last comment. After reading the TUIO2 spec. draft (which is still a draft so can be improved) is no more “reactable-centric” and they had tryed to define a protocol as much flexible as possible. The reason the spec draft is posted on these forums is because to have feedback from community, so if you think there are still things that do not match in it feel free to tell them rather than saying it doesn’t fit community, I believe Martin will be happy to get as many oppinions as possible. As some of the many improvements are how some new optional data is defined like oriented boinding box, outer countour of blob, interior contour lines, skeleton of a blob and its skeleton volume and messages for buttons inputs and a more OSC standard messages, where some of these new features where asked on this forum and personally I think it fits more the community desires or more data analysis output that LusidOSC seems also to lack.
I do not want to start a discussion on which protocol is better and fits best community needs (I’m sorry it seemed so, it’s not really my intention) but I’ll prefer to read comments on what is missing on protocols rather than say “it will need to be hacked on future” and “it’s reactable-centric”

I think this is interesting and important to talk about various alternatives, not to force change, but to get various ideas/opinions that’ll help with forward progress.

Adam, I think some clarification might be needed in what exactly you mean by ‘reactable-centric’. I think there may be some confusion by what your proposing, and what Martin is proposing in TUIO2. If I understand correctly, the idea is that TUIO is focused on the tangible (symbols, touch, cursor, and fiducials), while lusidOSC is a more general approach that’ll allow for expansion for things other than these tangible things? For example, if eventually the community were to start developing for head tracking, TUIO would need to be ‘hacked’ while lusidOSC would easily adapt to such changes? At this point, I’m only making this as an assumption.

I think one thing we’re forgetting is that NUI (nuigroup) is not about multitouch, but rather natural user interaction of a whole variety of future devices. Since this is the case, eventually we’ll need something that is more than a tangible only protocol. Of course this could mean we use TUIO and than another one for other devices; this is something to keep in mind though.

Adam, since you’re proposing a more community oriented approach, I think it would be nice if you can talk about specifics between TUIO and LusicOSC so we can get a better understanding of what you’ve changed and the reasoning behind it. This could lead us to a middle ground and understanding so this is a discussion rather than a one vs. the other argument.

To some degree, this conversation is a bit premature since we haven’t began to really identify some of the future technology of nuigroup and how they’ll relate to TUIO in the future. We also haven’t had the community involvement needed yet to really give us a healthy amount of information on what’s needed for an effective and efficient protocol from a variety of views.

Hello guys,
when we initially designed TUIO, we mostly thought about it as a communication protocol between various different table interfaces, such as the reactable and the tdesk, so I’d say it is not very reactable centric, but of course since we implemented it first within our own tracking software, it might seem so to the untrained eye.
We also published the protocol specification to the public domain with no particular intention to create an IEEE standard , but eventually there have been several people around here who apparently found the protocol and most probably the related TUIO client collection a useful toolkit for their own “multi-touch” efforts. We were of course glad about this widespread adoption by the community, and this is also the reason why I brought that discussion about the future of the TUIO protocol to this forum, and also in parallel to other people and places, in the hope that the community might find it again as useful as we do. After four years it has become necessary to think about a revision of the protocol, and I think that the current TUIO2 draft is already going into a good direction, it contains a lot of feedback from most current TUIO tracker developers and other people that have been dealing with the protocol layer in the past. I will start to implement the new specification after its final publication (which might still take a bit), and we will continue to provide our open source TUIO framework (reacTIVision,TUIO clients, TUIO server reference implementation) that will be also able to deal with both protocol generations. So on the client side you should be able to work as usual, but with additional features of course.

I want to make clear again that TUIO as such is in deed concentrating on the abstraction of an “interactive surface” and its “tangibles”, and I think it would be a bad idea to move it towards a general purpose “controller protocol”, because as a matter of fact, this is what the underlying OSC protocol already is anyway. And if you are looking for a VR style protocol, you should have a look at OpenTracker, which was specifically designed for virtual environments. We were actually considering to use it in the past ... http://studierstube.icg.tu-graz.ac.at/opentracker/overview.php

As for Lusid I really can’t say very much about it at the moment, version 1.0 seems to be basically a TUIO profile within a new name space and some parameters changed.
TUIO already provides the “custom profile” infrastructure in its initial specification, which would have probably covered the need for extension in this case I think.
In general the idea of having a more versatile and future oriented, as well as extensible protocol specification sounds of course like a good idea to me, but if Lusid was designed as such, I wonder why it already needed a major revision only a few weeks after its initial publication. But I know from my own experience, that it is of course difficult to foresee everything at the first place.

In my opinion OSC itself already provides the universal infrastructure for transmitting arbitrary controller data, but without any additional semantic information it is virtually impossible to interpret the transmitted data the desired way on the receiver side. This is why TUIO provides the basic semantics about symbols, cursors and blobs, which is what we are nowadays usually dealing with on an “interactive surface”.  I also think that TUIO2 will be much more scalable due to its flat profile, that organizes the various tangible “object” descriptors within the same namespace, which will allow for additional descriptors and if necessary also new “tangbles” in the future. Finally I also do not want to pretend to foresee all possible future scenarios within the current specification, because if doing so, one would also eventually clutter the specs with many features that are not used at all in the end. Therefore I’d rather stay down to earth and think about the currently possible and desirable features that a tracker also can implement nowadays and in the near future.

best wishes,
Martin.K

Thanks Seth and Martin, you both bring up some good questions.

LusidOSC is intended to work across a very large range of interfaces, including surfaces and objects, but also things like 3D gestures and GPS-based tracking, systems with a wide range of ID types and lengths, and can be extended to new technologies and setups without needing to create custom profiles (more on the importance of this later).  Saying that TUIO was “Reactable-centric” was not meant harshly; I was just pointing to the fact that TUIO is focused on “‘interactive surfaces’ and its ‘tangibles’”—an exciting, but fairly narrow range of new and emerging interfaces. 

Being from the Tangible Media Group at the MIT Media Lab, I certainly have a love for the various surface+tangible interfaces (reactable and audiopad being a couple of my favorites).  My main research goal is to make tangible interfaces accessible to people outside research labs who have creative ideas, but may not have the technical expertise or money to build larger expensive tables.  I originally planned to use TUIO, but after talking with a lot of people working on next-generation interfaces, I realized the community needs something a little different to really get more people involved.

LusidOSC looks a lot like TUIO (and credit is happily acknowledged), but the small changes allow for it to grow rapidly; both in terms of the range of applications and supported tracking systems.

The fundamental data in a LusidOSC message (unique ID, position, and rotation) is designed to work-by-default, regardless of the type of object, cursor, gesture, etc. since essentially there is only one main profile.  Having multiple profiles (as in TUIO), devices are segmented based on capability and use.  While this seems convenient at first, it also means that applications get segmented similarly.  For example, when a new custom profile or address is created to fit a technology, it won’t trigger any of the other events, even though there may be significant overlap in data.  With LusidOSC, any object, cursor, gesture, etc. will trigger an event in your application; as a developer it’s up to you to decide how you want to handle it, but at least you see the object.

Ad-Hoc data is supported in LusidOSC (without the need for custom profiles) so that developers of new systems can rapidly prototype ideas.  When the structure of how they wish to send their data has been decided, it can easily be amended to the LusidOSC specification.  Libraries will need only minimal changes for protocol updates (just adding labels for convenience to type IDs, but even that wouldn’t be required).  As developers, we greatly benefit from being able to experiment at the high level without needing to recompile libraries as we make changes.

There is also a philosophical difference between having a protocol created by a single person with community input and having the community itself creating the protocol. Both can work well, but I lean towards the community itself being in control.  In the spirit of that, everything said here about LusidOSC is open for debate end everyone is welcome to join in its development.

LusidOSC is very young, but we strongly believe that changes should be made as soon as they are agreed upon.  This is why, after only being released for a little over a month, we are looking at the next version. (Martin, you are definitely correct that it’s difficult to foresee everything when designing a scalable protocol). The next version of LusidOSC (v1.1) will make things a bit more explicit while still being completely backwards compatible. 

Based on the discussion so far, I imagine TUIO2 is a good next step for nuigroup.  As long as the community chooses to make the source open (for things like tbeta), we’ll have the flexibility to port it over to other protocols (such as LusidOSC) if interested.

Thanks for all of the discussion.  It’s great to see people interested in figuring out how to move all of these ideas forward!

Adam

Quick question. Is it right that Flosc converts TUIO messages to XML and then sends them to Flash application? Why this conversion? Why not XML only protocol?

XML is too bloated, really no one should even be using Flosc or the XML socket in flash. Thats what the flash binary socket is for. I have written a binary flash OSC socket that will remove the need for xml all together and will be releasing it shortly.

Daniel D - 04 March 2009 06:38 AM

Quick question. Is it right that Flosc converts TUIO messages to XML and then sends them to Flash application?

quick answer is Yes ;) and a little bit more

Daniel D - 04 March 2009 06:38 AM

Why this conversion?

‘cause the lack of an UDP/OSC connection in Flash, so a TCP connection is needed in a way or another. The XML socket is/was a quick way to convert the OSC data to an understandable data. The benefits of XML is that is easy readable, but in the other hand it adds a lot unnecessary information for just data (just think about all tags opened and closed to encapsulate data)

Daniel D - 04 March 2009 06:38 AM

Why not XML only protocol?

as sayd before, XML increase data size significatively, so althought it’s /has been a quick solution the best one is what already Pleh comented before. Hope to see those as3 clases soon ;)

But XML is a standard and understandable by all programming languages on all platforms. That is why it is so popular. And not something strange like OSC or proprietary binary.
MultiTouchVista happily uses XML to send contacts and image data for each frame without any lag.

XML is a way of Markup a text, but there’ll be a need however of an “standard” saying how should be markedup (i meant a DTD for example). OSC is an standard, and a binary socket which follows some specs and those specs are open and known mean is not propietary (I wanna mean with that that the uses of a binary socket doesn’t directly imply that it is propetary and/or not standard)

As programmer I can understand two mainly spread thoughts:
1) something that works doesn’t meant is the *best* solution
2) if something works do not touch it (lol)

Hopwever with your setup and your hardware you may not have any lag using XML for touchvista, but think about all cpu wasted for parsing and sending data, think about (just speculating) that something that can be encapsulated in 10 bytes of data you’ll need at least double in XML. From my point of view if thinking in something that should be as most fluent as possible and avoid overload then XML isn’t the best way. I’m not meaning XML is not a solution, just that is not the best one.

Im all for using standards, but I feel xml would just be sending too much redundant data down the wire. JSON would be a much faster standard and its used in all languages on all platforms too.