Hey nuigroupies,
I’ve been working on my own software for a while (touchlib is great, I know, but I like learning by doing) and I’ve recently got the finger tracking working to an extent.  Check out my video here:

For those that have worked this out for themselves, I am wondering what metrics you use to track blobs from frame to frame?  Also, do you track over several frames or just from current frame / previous frame?  Currently, I compare the current frame with the last and try to match blobs based on velocity then distance.

Hope someone can lend some ideas!

I’m just using distance.

I sort an array of possible matches based on distance.  Extras get a new finger ID and missings get removed.

Yours looks pretty speedy.  What platform are you targeting?  Are you dispatching TUIO?

Right now I’m targeting a PC as I’m not very experienced with development on any other platforms.  Sounds like we do something very similar with our tracking, as far as assigning new finger ids and removing ones with no matches.  I’m not dispatching the touch events outside of the program right now, but I think in the future I will try that… especially working with Flash.  I’ve done a lot of work with Flash in the past and I think it would be great for rapid prototyping.

Tracking looks pretty smooth, I do wonder how your IR LED illumination is placed …

Hey Falcon,
My screen is setup vertically and I have some custom IR lights I made (check my blog) that I place to the left and right just outside of the screen.  This eliminates reflections and tends to distribute the light well.  Even with the lights I made though, I already find that I would need more if I wanted to make my display bigger.  The video of me testing out the finger tracking was only done on a small 24” diagonal area.

I attached a simple diagram to show how I light my screen, hope it helps.

looks very clean

miketavius .. this is looking really promising.. could you explain the software a bit more? C++?… I really like the video above.. mad style

So I heard you got calibration working.  What method do you use?

My eyes went numb everytime I tried to read a SIGGRAPH calibration paper.  I can get the vanishing points, but I haven’t found any ways to use them to map the blobs in real time.

Thanks Mike.

The software is written from scratch in C++ using OpenCV, opengl for what minimal graphics I have thus far, and I use DSVL for video capture (which I found discovered via whitenoise, so kudos!).  I initially used just OpenCV for capture, but there is no way currently to adjust webcam settings like resolution and fps which really limits what you can do.  Anyways, I use OpenCV to recognize the blobs and then I wrote some code that does the tracking and calibration.

EDIT

I’ll do a detailed update later about how I do the calibration.

Very eager to hear about your process miketavius.  Without totally hijacking your thread here’s a brief of how I do it (no videos yet as I’m not working off a full table implementation yet) and would be interested to hear another programmer’s considerations.

Started off with OpenCV, dumped it because speed was not as high as I’d like.  OpenCV is very poweful but it does too much for what I wanted.
I’m using a custom blob tracker with a scanline flood fill algorightm that is very fast (and recursion free).  The filtering logic is done while floodfilling blobs so you don’t need to do run the entire image through chained filters.
I implemented this camera driver http://www.cs.cmu.edu/~iwan/1394/index.html for my unibrain which is nice as it provides an API for controlling brightness, exposure, gain, sharpness, etc from software (processing is done on the hardware).  I built an overlay opengl gui for controlling this with a Griffin Powermate (I can provide screenshots later).
For frame coherence I only look one frame back.  I currently don’t project based on velocity (I considered it but have not needed it yet) but I can see a few situations where that may be required, however I’d prefer to get a faster camera and keep the software as lean as possible.
I also implemented Zhang’s undistortion algorithm which I calibrated my camera/lens for.  This is done using a large lookup table (for speed) and eliminates any radial distortion (fisheye).  It still does need calibration.  Once calibration is in place, I’m not sure if you even need to do the undistortion.  I don’t think it’ll be necessary but I have it in place for testing as soon as I get my screen.