NUI Group » Wiki » Google Summer of Code 2008 Projects Ideas

Proposals guidelines

The student application period begins Monday, March 24th 2008 and ends Monday, March 31st 2008.

The ideas listed on this page are just that - ideas. They are deliberately somewhat vague, because you are meant to fill in the details. We don’t want to see project proposals which are basically cut-n-pasted from our ideas page. Don’t do that! If you do, you will be tossed out very early in the application sorting process. Be creative and extend our project ideas with additional description and your own feedback.

We don’t have exact template for your project proposal to use but we provide a list of information that we would like to see in it which includes: name, email, timezone, age, education, blog url, location, your project proposal, why you’d like to complete this particular project, are you an active NUI Group member, have you been working on multitouch, natural user interfaces, HCI and related topic before, the reason you’re the best individual to do so, what’s your academic and industry background, and of course open source development experience, explanation of your development methodology is a good idea, as well. Other details as you see fit.

Note for students: please remember that there is a 7500-character limit to proposals, so be prepared to supplement your proposal text with links to an external site, for example full version of your proposal in pdf file. You should also plan to provide an abstract of your proposal, including a brief list of deliverables, via the GSoC web app to ensure that your work receives sufficient review.

Additional resources to read:

Here you can find some additional documents and webpages that could be useful when writing your project proposal. Sometimes they link to other open source projects, but contain many useful guidelines which can be used when writing your NUI Group proposal.

Google Summer of Code™ 2008 webpage
Google Summer of Code™ 2008 Frequently Asked Questions
GSoC Advice for students
Google Summer of Code Knowledge Base
GSoC Announce group resources
Drupal’s Write a SoC proposal
PostgreSQL’s Advice to Students on Submitting SoC Applications

Propose your own idea

You really don’t have to use project ideas we covered below. These are projects that we have in mind to start working on or think that would be really useful for NUI Group community. Think of them as starters, even thought some developers think it’d be great to see them done. If you have your own idea, bring it on. We are always open for new innovative project ideas, so feel free to amaze us with yours. We’ll judge applications on their strengths, not on who wants the project done or who thought of it. However you might consider and find some of the project ideas listed below interesting. If you (as a student) have a new idea that is not listed here fill free to edit this project ideas wiki page and add your idea with your contact info.

Contact:

Feel free to contact us whenever you have any questions or f you want to use an idea, but you need more details.

You can find us at #nuigroup FreeNode IRC channel
look for nicknames: solydzajs, nuiman, whitenoize, xwolf and bauer

Join us at NUI Group forum at nuigroup.com/forums or our GSoC NUI Group discussion group at groups.google.com/group/nuigroup-gsoc to discuss GSoC & NUI Group project ideas and related topics.

You can also drop an email to our organization administrator at pawel (dot) solyga (at) gmail (dot) com

Mentors:

Christian Moore
David Wallin
Johannes Hirche
Mikael Bauer
Pawel Solyga

Projects Ideas

Some ideas might not be suitable for GSoC proposal because they are too easy and it’s development won’t last 3 months. Remember that you can still select this idea and combine it with other ideas in order to write GSoC proposal which meets GSoC requirements. In addition, some of the projects ideas might be to hard to develop during the summer by one student developer, therefore you can limit it and just work on particular parts. If you become interested you can always work on the rest features after GSoC - that’s our little open source world :-)

Touchlib

Mac OSX and Linux support - Port the existing codebase and techniques to more operating systems, and also easy to install downloadable bins for each OS. Package everything in an installer, and distribute packages in some linux distributions repositories.
Enhance Setup GUI - Create more detailed interface for setup and configuration of the vision engine and offer a more dynamic calibration system.
Wii Remote support for object/finger recognition/tracking - Wii Remote support so we could use the wii-remote as an input device, we are looking for Windows, Linux and Mac versions
Add multi-camera support - Implement stereo vision techniques and algorithms. Have tools for camera alignment and calibration.
Physical Object Tracking - Work on tag (fidicual) recognition, create a separate library which can be shared with Touchlib/OpenTouch and other project or merge existing library (reactivision) into Touchlib
Testing framework - write a testing framework for easy blob detection and tracking algorithms testing, write unit tests for touchlib, write automated tests
Windows Service support for touchlib (touchlib as a daemon) - add Windows Service support to touchlib so it can run in the background as system service instead of user desktop application, define and write communication layer between service and configuration application
Smaller tasks ideas:
- fps meter
- graphics in the “rectify” window, so that you can identify touches more easily, especially usefull if you only have a projector as a display, and potentially less contrast to see the spots in the rectify window
- hover detection feature
- abbility to control camera specs (for unibrain and ptgrey cams like brightness, exposure etc… similar what we can get with cmu firewire driver)
- autostart setup possibilty for exhibits. like start osc, start flosc and start flashdemo.swf (possible with batch files, more userfriendly approch)
- tide integration between Flosc, touchlib and you final Flash application with admin interface to configure everything

OpenTouch

Screen Calibration - Write multitouch screen calibration module or port it from touchlib.
Rewrite OpenTouch in Objective-C - For tight integration with Mac OS X frameworks and Cocoa. Utilize Core Image, Core Video for filters on USB or Firewire video feeds.
Create GUI for OpenTouch in Cocoa - Use Interface Builder to create a configuration frontend for OpenTouch have adjustable variables for video and blob tracking settings.
Add Diffused Illumination support to OpenTouch - Write an image processing code for Diffused Illumination multitouch sensing technique
Create project documentation, work on project webpage and tutorials
Add WiiMote support to OpenTouch - Wii Remote support so we could use the wii-remote as an multitouch input device (4 fingers limit), you can use DarwiinRemote open source project (http://sourceforge.net/projects/darwiin-remote/)
Improve blob detection and tracking algorithms in OpenTouch
Testing framework - write a testing framework for easy blob detection and tracking algorithms testing, write unit tests for OpenTouch, write automated tests

iPhone/iPod Touch applications

iTUIO - write TUIO simulator for iPhone (sending TUIO events from your iPhone to Desktop computer using WiFi, record your TUIO events for further replay, timeline support), use available open source OpenSoundControl Objective-C libraries
iMultiTouch - propose innovative multitouch iPhone application
Gesture Implementation- Write multitouch gesture recognition library for iPhone (defining your own gestures)
iGesture - write innovative iPhone application with heavy gestures support
Multitouch gestures support - add multitouch gestures support to existing open source iPhone/iPod touch applications

Other projects

Linux-based stand alone blob tracker - create a lightweight Linux-distro for ultra light pc that includes firewire-camera drivers, blobtracking software and sends the touch information trough ethernet or even better trough USB to the main computer. Also a controlling software is needed for every operating system to configure this linux-box. Also is important to determine how fast processor is really needed for tracking
MultiTouch mouse drivers for every OS - create mouse drivers for OsX, Windows and Linux so that they can be used without external mouse trough MT-display
On-Screen Keyboards - create on-screen keyboard software for OsX, Windows and Linux so no external keyboard is needed to operate regular programs trough MT-display
Quartz Composer TUIO patch - create new Quartz Composer custom patch for TUIO cursors and fiducials data, right now QC TUIO patch available at reactivision webpage has only fiducials support
~collab multisite collaboration using multitouch screens - write an application/framework which will allow for multisite collaboration on multitouch screens, what it means is that you can have many multitouch screen in different locations connected via internet or internal network and mt screen works as shared desktop with gestures support and synchronization between screens. You can have a look at croqodile project (croqodile.googlecode.com), TeaTime protocol used in Croquet project for syncing, solution for any platform will be most welcome, whether it’s Flash/Flex/AIR or OpenGL or any other platform specific software
work on MtLib multitouch Windows library - we have a project called MtLib in our codebase that hasn’t been released as open source yet, your work would be to release it, test it, create binaries, project webpage, write documentation and learn how it works. It is basically Windows multitouch library for Diffused Illumination multitouch screens written in C++ with DirectX support.
work on TUIO Simulator - extend TUIO Simulator which is written in Java to support timeline, multitouch gestures and easy way to add new protocols support
write new TUIO Simulator with timeline support and support for gestures and events recording so you can replay your events later on or save them to file and use later on for testing your multitouch app (look how iPhone Simulator works with gestures as an example)
work on libavg project - add TUIO support, FTIR screens support, GUI, create installers for Windows, Linux and Mac OS X
work on openframeworks project - add TUIO support, utilize openCV for blob tracking, port to linux, define specialized library or addon to natively using multitouch.
http://www.openframeworks.cc/documentation
WiiMote multitouch framework - create a framework for writing multitouch applications such as interactive whiteboard using WiiMote under Mac OS X (DarwiinRemote project might be helpful), write some example applications
Work on WiiMote Whiteboard project - create new example applications, port WiiMote Whiteboard to Linux and Mac, write unit tests and create installer with binary distribution
Improve blob detection and tracking algorithms - write library for blob detection and tracking so that it can be used later with different projects ex. touchlib, OpenTouch
Your innovative multitouch application - propose new innovative multitouch application you would like to work on (interaction in 3d environment, creating scenes, documents), amaze us with your ideas
work on WiiMoteTUIO project - example applications, bug fixing, create a cross-platform solution, write test and documentation
Automatic projector calibration - propose a project for automatic projector calibration used with multitouch screens in DI and FTIR setups
Multitouch gesture library - write multitouch gesture recognition library with support for gestures that could consist of several strokes like /\ + - = A, simple one finger gestures like mouse gestures and multitouch gestures (zooming, pinching, rotating and many more), use fingerworks gesture dictionary for example gestures (www.fingerworks.com)
Multitouch applications on Mac OS X - propose and write multitouch application for Mac OS X, using TUIO protocol for events and new Leopard graphic frameworks like Core Animation, Core Image, Core Video, QuickTime etc.
Firefox TUIO extension - write Firefox TUIO extension which maps TUIO events to JavaScript events and adds gestures support in Firefox using same JavaScript extensions used in WebKit/Safari introduced in iPhone SDK
Research multicomputer use for image processing - work on distributed blob detection and tracking solution, dividing computation between many machines, this would be really useful for big installations like multitouch walls and with many cameras, test your solution and integrate with OpenTouch or touchlib
Multitouch Phun - write a multitouch 2D physics sandbox application (look at project Phun to see how it actually works with mouse, http://phun.at/),you can use existing open source physics libraries and TUIO for multitouch events
MB Pro and MB Air multitouch trackpad application for Mac OS X - propose and write multitouch application which supports MacBook Air and MacBook Pro multitouch trackpad gestures or extend existing open source project to use the power of intuitive trackpad gestures
reactivision project - work on better multitouch finger detection and tracking in Reactivision project
GPU accelerated blob detection and tracking algorithms - write GPU accelerated blob detection and tracking algorithms for DI or FTIR techniques, you can use CG for NVidia or compare usage of GPUCV vs OPENCV

Multi-Touch Enabled Clients Proposals (Touchlib/OpenTouch Demos directly using library or via TUIO event and TUIO Simulator for testing)

Touch OS - A rapid prototyping environment for testing multi-modal applications (Flash/WPF)
Touch UI - A framework for multitouch oriented components and objects
Touch API - A standardized communication layer (XML/Binary Socket or Internal format)
Touch Web - A multitouch web browser in Adobe AIR using WebKit engine or OpenGL version using the Mozilla codebase and uBrowser OpenGL Gecko port (www.ubrowser.com)
Touch Earth - A multitouch earth browser in Adobe AIR/Flash/Flex using ModestMap and with support for different map providers including Yahoo , Google Maps, etc
Touch Gestures - A multitouch gestures recognition library for Adobe AIR/Flash/Flex
Touch App - Propose creative and innovative multitouch client application (OpenGL/Cooca/WPF/Flash etc, you can pick a technology and language you want to write it in)

Related Resources

Touchlib - http://touchlib.com
OpenTouch - http://opentouch.googlecode.com
TouchAPI - http://touchapi.googlecode.com
Reactivison, TUIO and TUIO Simulator projects - http://reactable.iua.upf.edu/?software
Multitouch project - http://multitouch.sf.net
OpenTable project - http://opentable.googlecode.com
Libavg - http://libavg.de
iPhone SDK - http://develop.apple.com/iphone
WiiMoteTUIO - http://code.google.com/p/wiimotetuio/
WiiMote multitouch - http://www.cs.cmu.edu/People/johnny/projects/wii/
WiiMote multitouch forum - http://www.wiimoteproject.com/
WiiMote WhiteBoard project - https://sourceforge.net/projects/wiiwhiteboard/

View Google Summer of Code 2008 Projects Ideas