I’m planing to start a project today using the PMD CamCube 2 for multi touch and head tracking.
The display will be a stereoscopic 3d projector setup.
For those who do not know, the cam cube is a depth sensing time of flight camera with a resolution
of 204x204 pixels and a depth resolution of ~1cm.
I made a screenshot today:

As you can see there is alot of noise in the picture, especialy in the background but in principal
you can clearly detect important regions. Also my setup will be in a controlled environment so
in the end removing the background should be the least problem.
I’d like to hear some thoughts on this project. Are there any experiences using a similiar setup?
Basicaly the images look quite similar to those of rear projection setups so i might get this to work
with only minor modifications to the existing nuigroup software.

I wrote some filtering and body detection. This is what i get up to now.
The depht resolution would be higher but i had to map it on 8bit grayscale.
I’m going to combine the low res camcube with a high resolution camera to get better results on the fingers.
http://lh6.ggpht.com/_MOyeVmMdA3g/SyJ3qHz7mdI/AAAAAAAAADI/yQ8YbPRMBHc/filtered.jpg

hey there,

any progress to your project? i’m very interested.

marco

I’m making constant progress but nothing realy noteworthy atm.
I wrote a zBuffer ragdoll renderer that i want to use to check results and try out variations.

Also I am working on skeletonization of the polygon to find out body regions in the shape.
Working with euclidian distance transform. I should be able to get the skeleton with watershed and differenciation.
http://corvusmedia.de/korbi/vortrag/skel.jpg

Awesome stuff.

How much is the CamCube?

Are you going to make your code available?

Camcube is 8000 euro atm but it will probably drop in price a lot over the next year. I heard something about mass production and 1000 euro. Time will show.
I am not sure what to do with the code yet but i’m a big fan of open source so i think if this project gets somewhere i will put it on sourceforge.

wow! 8000 euro (or even 1000 euro) is much for a DYI project…

It is my final year project at university. Couldn’t afford to play with such stuff otherwise.
I even didn’t have to buy a camera because there is a full chair in informatics here working with TOF cameras. =D

can you tell me the applications in which you use or plan to use the cam I mean possible scenarios where it might be very helpful?

At your university what kind of apps they had or plan to develop?

I build a stereoscopic 3d projector before and want to implement head tracking for better immersion and motion capturing for interaction with the environment.
I’m a student of physics so I’m not directly involved with any of those informatics people, but as far as i know they do medical stuff with the cameras - patient placement and touchless controls for OP. Such stuff.

do you have some links of the department doing the projects?

http://www5.informatik.uni-erlangen.de/

My zbuffer renderer works realy smooth now:

Now i will work on a modell that fits me.

I just wanna say : wow!

Solved some issues with the skeletonization
http://www.youtube.com/watch?v=7uQvUp0bms8