I would like to share my idea for Google Summer of Code for NUIGroup. In short, I am trying to develop an application framework that will:
a) work on different platform and devices,
b) provide the flexibility to build applications with most natural and meaningful gestures, and
c) portable: minimum or no installation
Now, “different devices” includes different types of devices like Microsoft Surface, Multi-touch Tablets & Desktops, Mobile Devices (i.e. iPhone, Android) and “different platform” refers to different types of operating systems like Windows 7, Mac OS Snow Leopard, iPhone OS, and the like.
Although most devices and platforms have their own technological distinctness, they all support a common platform – browser based web applications. Also, web applications are highly portable as they don’t need installation. And, most popular web browsers (i.e. FireFox, IE9) now provides support for touch and mobile device browsers (i.e. iPhone - Safari) also provide touch events.
Finally, lets look into possible applications:
1) Software Designing tool: Sketch enabled UML Designer (i.e. SmartUML),
2) Molecule Bond drawing: could be a useful tool for chemistry students (I saw an app for iPhone… lost the url)
3) Different drawing applications
4) Planning tools
5) GIS data presenting tools
6) Collaborative tools
If we study the requirements, the graphical & programming challenges can be meet by new HTML5 features. But, for user interactions, research studies show that often pre-defined gestures are not good enough and application developers need to create/define new gestures that is natural & meaningful to particular application context.
Here, we have two issues: First, existing gesture recognition toolkits are build in a way that only locally installed applications can use them. So, we can’t directly use them in web applications. Second, we don’t actually have any nice and easy way to define new gestures. To address this issue, I propose a flexible and extensible gesture definition language. I have been working on this for sometime now and you can see some examples here: http://bit.ly/9BuZPG. Its in early stage of development but I have some demo that I will upload shortly.
Please let me know what do you think. A more detailed proposal is available here: http://bit.ly/anc39L