3D User Interfaces in Video Games with the Bespoke 3DUI XNA Framework
Joseph LaViola and Paul Varcholik of the University of Central Florida demonstrate a series of simple video games that utilize concepts from 3D user interfaces and virtual reality. The demos let users play music with virtual instruments, battle Boba Fett, dance, and play 3D Pacman. These demonstrations have been built using the Bespoke 3DUI XNA Framework, an open-source software library based on the Microsoft XNA Game Studio, for developing games and simulations using 3D user interfaces. The framework’s goal is to make it easier for game developers and hobbyists to create mainstream games that give users rich, interactive experiences that can mimic the real world or provide magical, larger-than-life experiences. The key feature of the framework is that it provides a set of software components that are essential to supporting spatial 3D interaction, such as stereoscopic rendering, 6DOF optical head tracking, Wiimote 3D motion controller support, and an extensible 3D gesture recognizer.
Can I urge we don’t use this? Lux is a good enough framework, comeing out sometime in *cough**cough*, which is probably before this will com out. Lux is cross platform, this is not. I just think we need to get the ball rolling on cross platform toolkits(C++, Python, Lux, openFrameWorks).
With a new paradigm shall come an old and proven concept: Freedom
I’m Paul Varcholik, author of the Bespoke Multi-Touch Framework. I started research Multi-Touch technology in August 2007, and have built two hardware platforms thusfar. The first prototype is what you see in that video (self-funded). I’m pursuing my PhD at the University of Central Florida, and my lab has adopted my work. This has provided funding for the second hardware platform pictured below. This new platform is brand-new (delivered last Thursday) and we’re still outfitting it. As to the libray—the main focus—it’s written in C#, is open-source, and is decoupled from the presentation layer. This allows the use of GUI systems such WPF, WinForms, and XNA. I’m also working on a network gateway to allow inter-process communication to non-.NET environments. I’m looking at OSC and TUIO, as I understand this is what TouchLib is using. Does anyone have any feedback in this regard? What are people’s experiences in writing apps against OSC/TUIO? Is this the direction I should head in for the gateway? Additionally, though I’ve targeted Windows as the chief platform, I have a mind to port the library to Mono, opening up Linux as a platform. How many NUI Group users out there are using Linux?
Thanks for the interest in the library. I will be making it available for download very soon.
Thank you Paul for your input, and welcome to the forums. I myself am a GNU/Linux user, and I have always had one question about frameworks like this. How is WPF, XNA, WinForms, and all the other new names for Shiny but old and outdated DirectX any better the proven and time hardened cross platform standards like OpenGl? You could target so much more just by writing it in C++, with OpenGl as your graphics library. This is what Lux(our community driven framework) looks to do. I do not see any reason to program C# over C++, and I see many reasons to program C++ over C#.
There are a good number of us using GNU/Linux, I couldn’t give you an exact count.
Thank you Paul for your input, and welcome to the forums. I myself am a GNU/Linux user, and I have always had one question about frameworks like this. How is WPF, XNA, WinForms, and all the other new names for Shiny but old and outdated DirectX any better the proven and time hardened cross platform standards like OpenGl? You could target so much more just by writing it in C++, with OpenGl as your graphics library. This is what Lux(our community driven framework) looks to do. I do not see any reason to program C# over C++, and I see many reasons to program C++ over C#.
There are a good number of us using GNU/Linux, I couldn’t give you an exact count.
Best regards and hope I don’t start a flamewar,
Alex
Hi Aex,
No flamewar here, you’re certainly entitled to your opinion. I’m also a Linux user—that’s where I got my start as a software engineer (not counting my first gig as a DOS real-mode (below 640k) programmer). To your question, “How is WPF, XNA, WinForms… any better than OpenGL?” I’m not sure there’s a truly better/worse yes/no answer when choosing APIs—I think it’s largely a matter of opinion and preference. That’s not to say that you can’t do feature, memory, platform, stability, security, comparisons. But sometimes, the choice of a development environment is just that—a choice. I started coding in C (1994), then moved C++. In 2003 I moved to C# as my preferred language—and I’m betting that sometime in the future I’ll change again. For rendering engines, I started with OpenGL, then DirectX, then Managed DirectX, and now XNA. For GUIs, I started with ASCII, then the Windows SDK, then MFC (sigh) then WinForms, then WPF. Is C somehow worse than C#? Is OpenGL worse than XNA? Not necessarily. I just find myself t be a more productive coder with the current envionment I’ve chosen. The best response I can give is “Try environment X and see if you like it”. Or, if you’re happy with your current environment than stick with it. For me, when I tried C# I loved it. Likewise for XNA. In the late 90s I taught classes in Linux and C at a local technical college. Now I’m teaching classes in C# and XNA. I could give you a laundry list of reasons why I like this new environment better—but they’d just be my reasons, and may not necessarily suit you.
In the context of Multi-Touch, and platform independence—an IPC gateway is a way of supporting more environments than what the native development platform targeted. I’m fairly certain you’ll provide such a thing in Lux—because not everyone codes in C++ and will natively reference your library. True?
Welcome… your demo’s look really good. Can’t wait to try it out..
OSC and TUIO is definetly the way to go if you want to interface with non-windows platforms, unless you can find some way of to emulate how touchlib works
There is another good framework around called MultiTouchVista, which i would see as the basis for a Windows MultiTouch Framework, maybe you cooperate with them and provide a unified windows framework?
My personal preference goes out to a MultiTouch framework which makes development in WPF easy as this is my language of interest, so for now i’m concentrating on WPF with MultiTouchVista
WPF really makes coding/design on windows alot easier..
Very nice work Paul and thank you for your contribution. Looking forward to downloading and trying it out. I’m just getting up to speed with C++ and Visual Studio and WPF.
I completely agree with you on in the respect that we all have our own opinion, and while I might like OpenGl you like XNA. I think my main reason for this is I don’t want to leave Windows and Mac users in the dust, so I picked a standard. XNA is certainly younger then openGl, but I have never tried it. Same for C#. I am sure you have a reason, regardless of what it is. I personally do not code C++, though I do respect those who do purely becuase it is cross platform(when coded right). Personally, I do Python.
We will be targeting AS3, C++ and Python at first with Lux, and more platforms as requested.
One advantage I could see to C#/XNA over Python/C++/OpenGl is you never have to worry about Microsoft breaking your code in the next release, because it won’t happen for another seven years!
PaulV, any reason you’re not using TouchLib as the basis, and use a C# for the rest? I expect you’ve written your own video processing/filters. As Touchlib is cross platform it would have saved you alot of time back then, or had it to do with commercial reasons?
One advantage I could see to C#/XNA over Python/C++/OpenGl is you never have to worry about Microsoft breaking your code in the next release, because it won’t happen for another seven years!
Lol, actually Microsoft announced Windows 7 for somewhere in 2009/2010. It will be based in Vista ofcourse.
They said that the long development time for Vista was an error, and that they aren’t planning any “mayor” new released like Vista anywhere in the near feature..
PaulV, any reason you’re not using TouchLib as the basis, and use a C# for the rest? I expect you’ve written your own video processing/filters. As Touchlib is cross platform it would have saved you alot of time back then, or had it to do with commercial reasons?
I also work on touchlib replacement. The main reason is too little information about surface. You can’t do things like “Shadow Box” with TouchLib.
I’m obviously a bit biased here, but I think the right way to choose a development framework is to look at your end goal in terms of user scenarios.
If the goal is for end users to get multi-touch capabilities integrated into their existing apps, you should target the framework that the majority of those existing apps are built with. In most cases, this will imply providing low level C/C++ APIs. The work Peter H. has done with the open source MPX project is a really interesting example of this.
If the goal is to speed the creation of brand new apps with UI designed from the ground up to take advantage of the mutli-touch/multi-user/object-recognition capabilites of a new platform, you probably want to invest heavily (though not exclusively) in targeting a high level framework. Our work with WPF is an example of this… by providing reusable and highly customizable UI controls like ScatterView, apps can focus on implementing domain-specific concepts and get to a deployable state much faster.
(If there is an additional goal of being able to run on any OS… well that’s not something I’ve given much thought to but maybe something like Mono/Moonlight play a role)
It’s *really* hard, but the best platforms will figure out how to strike a balance between multiple goals. When I give presentations about the Surface SDK, one of my first slides is something along the lines of
“What do great development platforms have in common?
1) Enable but don’t require re-innovation of application fundamentals (hit testing, input processing, UI rendering, etc)
2) Enable but don’t require the use of Rapid App Development (RAD) practices (reusable controls, reusable animations, WYSIWYG UI design tools, etc)”
Those points aren’t specific to multi-touch but they help to guide us towards generating a scalable platform for the future.
