NUI Group - Open Source Interface

[GSoC 2012 Proposal] Open Source Infrared multitouch frame

2012-03-25T12:47:53-05:00

Name: Alexandr Pirogov
E-mail: a.g.pirogov@gmail.com
Location/Timezone: Saint-Petersburg, Russia, GMT +3:00
Age: 23
Education/Qualifications:
- Bachelor’s degree in Information science and computier engineering from Saint-Petersburg Polytechnical University
- Currently a second year of Master’s student at the same university
Academic and Industry Background: I’ve been working past 2 years with embedded systems with FPGA’s, developing HDL interfaces for memory and touch-screen controllers and processing acquired data on NIOS II processor using C/С++.
Open source development experience: So far no exp in open source projects.
Development Methodologies: Depends on company politics :) In this particular project, I’d go with agile methodology with regular feedback from menthor
Project Proposal: Open Source Infrared multitouch frame
Main goal – develop an easily scalable IR multitouch frame
1) Develop scheme of the board with IR tranceivers, microcontroller and USB interface.
2) Program controllers to collect data from tranceivers and send it via USB interface.
3) Prepare driver for computer to collect and process acquired data.
The main idea of this project to share all the details and let everybody else make the same frame.
There are few companies, who produce these kind of devices with true multitouch interface, but their cost is extremely high, while cost of components can be 10 or 100 times lower.
Another pro is IR frame can achieve much bigger size than capacitive touchscreens, that can let you turn your home widescreen TV in a giant iPad. )
The reason for picking specific project: It will be base device for further development. I already have some ideas for implementing this frame in daily routine.

Activity level within the NUI Group Community: Recently registered.

EDIT:
Full version of proposal document
http://dl.dropbox.com/u/56367755/Proposal%20GSoC%202012.pdf

[GSoC 2012 Proposal] Moving CCV Forward (+CCA and Fusion)

2012-03-26T10:11:23-05:00

Name: Scott Halstvedt
Email: ,
Location/Timezone: Missoula, MT (GMT-6)
Age: 21
Education/Qualifications: 3rd year Computer Science student at the University of Montana (with a focus on Artificial Intelligence). I’ve been taking CS at a college level since 2007, and learning bits on my own for longer.
Academic and Industry Background: Studying CS at the University of Montana and briefly at the University of the Pacific, CA, USA and Rocky Mountain College, Billings, MT. Recently I have been trying to focus my education on applying the most powerful algorithms (AI and machine learning) to bridge the huge interface bandwidth gap that exists between humans and computers. I took an overview of AI last fall, and I’m currently enrolled in an implementation-heavy graduate machine learning course.
Open source development experience: Still pretty limited. Although I have submitted some patches to small open-source projects before, I got my first real taste of a project last summer with SoC and NUI group, which was certainly a learning experience that I think puts me in a lot better position this year to understand how to approach such an undertaking this year. This year I’d like to select a more specific goal that is doable by one developer (last summer was a little ambitious, although I gained an enormous amount of experience from it).
Involvement with NUI Group: Summer of Code ‘11 student, Fusion. School has prevented me from working on that very much outside the summer, but I very much enjoyed my experience working with NUI and would love to do it again now that I know the ropes.
Development Methodologies: I like formality. I tend to spend a lot of time conceptually organizing the project before it starts to take shape, and I like to keep doors open to future functionality. I enjoy sketching out the future more than refining the past. I work effectively even in situations with limited foreknowledge (I love reading papers about algorithms).

Proposal: Moving CCV Forward (+CCA and Fusion)
The CCV framework is clearly the flagship of NUI’s software offerings to the community. It can be found all over the internet, from the “software tools” sections of numerous academic papers to tech news websites writing about how easy it is to build a multitouch table with NUI Group’s resources. CCV fits in situations where ease of use and setup are paramount; if someone wants to create an advanced computer vision solution, they will use OpenCV and write their own client, but many people do not want to (and should not have to) reinvent the wheel. So, CCV is the established player, and we need Version 2.0 to be more than a feature-parity rewrite. We need to show something really really cool and I think we’re well on the right track. Last summer Jimbo Zhang did a lot of the hard work to migrate CCV’s engine to the Movid framework, giving CCV a modular architecture and an unfixed, drag-and-drop configurable pipeline. This flexible but accessible configuration interface fits perfectly with the target audience of CCV, and the modular architecture will allow us to add features while maintaining simplicity and without unmanageable UI growth. I’d like to spend the majority of my summer working on CCV2, and ship it in style.

1) Continue working on CCV2 and the nuiFrameworks idea to migrate to the modular, pipelined Movid backend. I have a great deal of familiarity with the organization of the Movid framework from my work with Fusion last year. I’d like to ship a working beta version of CCV2 by the middle of SoC, and a final version that is at feature parity with CCV1.5 by the end of the summer.

2) Neural network blob tracking in CCV2. I just implemented a neural network classifier for a machine learning course, and the OpenCV project provides a library of machine learning algorithms that might be applicable to this part. This would give the new CCV some new features to show off.

3) The idea of nuiFrameworks and the Movid backend lay the foundation for the future of NUI’s software offerings. The Fusion project I worked on last summer is my idea of where things are headed, but the groundwork is not laid yet and we can’t jump to the finish line. Continuing to evolve nuiFrameworks to the point where it can be used in several applications is the ultimate goal of this proposal, after shipping CCV of course. This unified framework will allow a tighter coupling between CCA, CCV, and CCW (something I struggled with when trying to implement the concept of fusion) and lend the benefits of Movid such as a threaded architecture and drag-and-drop configuration to CCA and others as well.

I’d very much appreciate the community’s feedback on this proposal. Are people more interested in a neural network classifier implementation, or perhaps more time spent polishing nuiFrameworks?

I have some other, crazier proposal ideas too that I may post later today or tomorrow, but I think this is the project in which I could help the community most this summer, given my familiarity with the framework already.

Thanks for reading!

GSoC 2012 - 3D tracking of hands using Kinect

2012-04-04T17:34:54-05:00

[youtube]http://www.youtube.com/watch?v=Fxa43qcm1C4[/youtube]

Hi, I’m sorry but I’m very late in discussing the idea with you - it’s just 1 day left but I’m sure I can write a brushed up proposal if you like the idea I’m going to write. Also, I will write about my experience and skills in the next post while you go through my idea.

I propose 3D tracking and articulation of single hand using kinect device. Human hand pose is represented by 27 DOF’s and finding these DOF’s is essentially the problem. To do this, I will have a hand model (synthetic hand) in OpenGL represented by these 27 DOF’s. By means of rendering I can get the depth map of this synthetic hand. On the other hand, kinect will give me an RGB image and a depth map and using skin segmentation techniques I will extract the relevant portion containing the hand out of both the RGB image and the depth map. This depth map is compared with synthetic hand’s depth map and discrepancy between the two is found. The 27 DOF’s of synthetic hand correspond to the human’s hand when discrepancy is ideally zero. Finding such values of 27 DOFs of synthetic hand is essentially an optimization problem which can be solved by using particle swarm optimization and the process can speed up if GPU acceleration is used, giving us real time tracking of hand.

This is basically the implementation of a paper and would give us results like in the video above.

I plan on extending the project for the case of two hands with several complicated cases once I’ve done it for the case of one hand. I’ve already done some very basic work on skin segmentation and set up of basic framework using opengl, openkinect and opencv but since the research paper requires a lot of work (skin segmentation, modeling of hand, GPU acceleration, implementation of variant of particle swarm optimization) i think this can be a potential GSoC project. And if time allows, I will definitely extend it for the case of two hands for GSoC.

Uses: There are many! one is to provide interface to 3D modelling applications like Maya, where you can use your hand to model 3D characters instead of tedious job of using mouse to modify each primitive in the model.

[Kivy ] Building a GUI builder for Kivy using Kivy

2012-04-04T18:19:26-05:00

Hello!

I am Pranav Rao, a 20 year old undergraduate student majoring in Mathematics and Computing at Indian Institute of Technology, Kharagpur, India.
After lurking around in the IRC channel, developing a game for the Kivy contest #1, and discussing some implementation details with +t-shirtman,
I now present the first draft of my proposal.

Along with the pdf of the proposals, I have attached 2 wireframe mock ups of the UI of the GUI builder tool. Please refer to my Proposals pdf for
detailed implementation details for both the UI and the backend for the app.

I really need help polishing this, especially part 2.3 of the document. Apart from this I have left 2.4 open to accommodate any feature requests
from the Kivy community here.

Do let me know if I should post the text version of the pdf here if you cannot open attachments. I tried once, but the formatting was all weird.

Thank you so much for your time,
Pranav.

[Kivy] Widget synchronization between processes

2012-03-22T11:45:28-05:00

Hello!

My name is Jason Walsh, I’m 19 years old and finishing my freshman year at Case Western Reserve University. I’m interested in working with Kivy for Summer of Code 2012, and specifically the network serialization idea from the ideas page. Inter-process communication is something I’ve run into in personal projects and it’s made me interested in efficient and effective ways of sharing state between processes. Beyond that, the image of being able to throw objects from one device to another is really enticing and I would love to be able to make it happen. I’m very self-motivated on projects that interest me, and I think I’m an effective problem solver and programmer. If I’m accepted to work on this project over the summer, my general plan of attack would be something like this: 1) share a single widget between two Kivy processes running on the same local machine; 2) share a single widget between two Kivy processes over a network; 3) share an arbitrary number of widgets between two Kivy processes over a network; and 4) implementing natural, intuitive interfaces for pairing a process with another Kivy process, choosing widgets to share between the two, and what it would look like when those widgets were being shared.

You can find more information about me at http://jasonwalsh.us, some code I’ve written at http://bitbucket.org/jasonwalsh, and I’d be happy to provide a resume or transcript. I look forward to hearing feedback on my idea, and hope the project would be interested in having me write a more formal proposal to submit.

Thanks!

Mobile assisted user interface for PC navigation and gaming

2012-04-05T13:04:40-05:00

Introduction
Keyboard and mouse are of great use in day to day computer related work,but when it comes to gaming they are not so handy. So we look at options like XBOX 360, Nintendo wii, PS3, etc. Although these things are fancy, they are not quite affordable. These gaming console consists of limited set of keys which are binded to specific actions, motion sensor so we do not physically need to press the arrow key, just we need to tilt the controller. Many of them are wireless.
Almost all mobile consists of bluetooth so they allow us for wireless connectivity, they also consist of sensors like accelerometer/gyroscope which accounts for motion sensor. Mobile also consists of limited set of keys which can be binded to specific action. So we are able to use mobile itself as game pad.
The use of such application is not restricted to gaming only we can use it for navigation of windows also. Example if we send active co-ordinates of our touchscreen to PC. We can use our touchscreen as Computers Touchpad. Computer should know what mobile is sending and what actions to perform for this a powerful configuration system is required.

Platform planning to target: Android

Are their any place for such project in GSoC this year??
please give your suggestions.

[GSoC 2012 Proposal] Arduino framework for physiological sensors with a machine learning twist

2012-04-04T15:58:06-05:00

Name: ***
Email: ***
Location/Timezone: Helsinki, Finland / GMT+2
Age: 24
Education/Qualifications: I’m currently working towards two masters degrees: an MA at Media Lab Helsinki (Aalto University School of Art and Design) in New Media (focusing on physical interaction and user interfaces), and a MSc in Engineering continuing my BSc in Computational and Cognitive Bioscience with a minor in Computer Science (Aalto University Schools of Electrical Engineering and Science and Technology)
Academic and Industry Background: I started recently working as a research assistant at Media Lab Helsinki in the Sound and Physical interaction research group. Our focus is on developing new user interfaces based on sound, haptic feedback and new kinds of physical interfaces. I have also worked as a research assistant at the University of Helsinki since 2010 in the Brain and Music team at the Cognitive Brain Research Unit. Working full-time during the summer months for GSoC won’t be a problem though, as both positions are part-time and activity is pretty low during the summer months.
Open source development experience: I have been using open source libraries for my hobby and work projects, but haven’t contributed before.
Involvement with NUI Group: I’m looking forward to getting involved with the NUI group, as the work is very related to my academic direction and general interests.

Proposal:
I would like to propose and get feedback on a project related to physiological and affective computing.

My idea is to make a framework for arduino that includes open hardware designs and schematics for monitoring various physiological signals. A lot of this material is already available online and it is a matter of gathering the right information. As a reference I can mention Pulse Sensor http://pulsesensor.com/, although it is documented extraordinarily well.

Apart from this, the framework would include libraries for analyzing and interpreting the various physiological signals, working directly with the proposed hardware designs. The signal analysis would be done directly on the arduino, and resulting data transferred through the serial protocol so they can be easily integrated into most programming environments to be used as input.

During the summer I plan to include at least heart rate (photoplethysmography), skin conductivity and breath rate sensor designs and analysis modules, providing the reference structure for modules to aid in further development.

Adding to this I plan to also implement a more experimental twist to the system. This is a machine learning (supervised learning/regression) approach to physiological data, that can be sometimes hard to interpret in a meaningful way. Gathering data for the learning phase would be implemented so that the arduino can receive target numbers through the serial interface and return the gathered data with this target included. The machine learning model will be trained on a PC on what will be a python application employing machine learning algorithms from open source projects. Plan is to then transfer the resulting model back to arduino, so it will then output the values returned by the model for new data. This allows the user to experiment with combining physiological data and constructing a meaningful output, mapped to for example emotion, audio features, musical styles, etc.

I’d love to hear some feedback for this project. If you think it’s too far from the nuigroup focus, I might still try to work on another proposal that would be more physical(instead of physiological) in nature.

Reminder GSoC 2012 Student applications Deadline is April 6th

2012-04-05T04:17:18-05:00

Hi all,

Please remember to post your project proposals via official Google Summer of Code web app ( http://www.google-melange.com)
Student applications deadline is tomorrow (April 6 19:00 UTC).

Best,
Pawel Solyga
NUI Group

[GSoC 2012 Proposal]- Kivy , GUI Framework

2012-04-02T17:18:41-05:00

Hi there,

I want to make a GUI framework for Kivy using Kivy. It will be a lot useful for newbies in Kivy and its all in Python.

Rough idea for it’s UI and functionalities are

-User can make a new project or can save this project and open saved projects.
-There will be two different tabs, one will be Design Tab and second will be Code Tab.
-In Design Tab, there will be a hierarchical model of parents and children for widgets.
-One parent can also have more than one child, as we needed. But there will be only one Grandparent for hierarchy.
-User has to give name to every widget when it’s created. (In code that name will be name of that widget(class)’s object(instance))
-There will be different modules for Timer, Animations. They won’t be in hierarchy but could be bind with any widget.
-Different attributes of widgets could be set from down side part in design tab. Option to overwrite class of widgets to overwrite its methods [Events] will also be available.

-In Code tab, Code of Design Tab will be generated automatically. “.kv” file will also be generated automatically. User can swap between Design tab to Code tab anytime.

To make UI, Design tab would be useful, and for detailing functionalities, Code tab could be useful.

Through this framework, Kivy could be used by anyone so easily.

Please let me know what you think of this idea. I want to get some feedback from the community before starting the official proposal for GSoC.
You can ask queries on IRC also, lurking on IRC as metalfinger.

[GSoC 2012 Proposal] Open Source Conference Call System

2012-04-04T21:33:43-05:00

I know this is kind of late, but I was wondering if there would be any interest in a conference call application for community core audio.
I was thinking it would take advantage of other oss like mumble, which is a client/server voip app, and CLAM for audio manipulation and analysis. So one app that could be extended a number of ways:
- Be able to have NLP, sentiment analysis of all participants
- Autonomous or semi-autonomous mediation
- Dominance detection
- Topic modeling
- Work with ccv to capture non-verbal behaviors
- easy setup to collect data for social scientists
- should work with web, smartphones. just need to login to a server
- because its networked, explore communication with other networked devices? If all the devices in the home are on the network, then one could communicate to the TV or lighting systems
- virtual agents like Siri

There are lots of interesting ways this could be extended. A lot of this work is related to my research (I am a 2nd year PhD student at CMU). If I could get some feedback on what would make this appropriate for a NUI GSoC proposal, I can quickly write one up.