Hi everyone,

I want to thank all of the members of the NUI group for being so generous with information and assistance!

We are prototyping exhibits for our new Science Center, and have learned a lot about multi-touch thanks to all of your posting and discussions.  We’ve made good progress on our own table and would like to show it off and get some feedback.

We have a DI setup using a BenQ MP512 ST (short-throw) projector

BenQ has a nice “projection calculator” web-app that lets you calculate the distance between the projector and the screen and the screen size (make sure you select the right projector in the pull-down menu).  Very useful!

This website was very helpful for figuring out the table/projector/mirror layout.

This images shows the Microsoft webcam, but the Firefly MV is in the same location.  Note the projector is mounted upside down to an old slide-projector stand that allows adjustments.

We started out with a Microsoft LifeCam VX6000 and, based on the feedback here on the NUI forums, quickly graduated to a FireFly MV from Point Grey Research.  For those of you who are hesitant, we found that it is well worth upgrading.  The slow frame rates of most webcams are just not worth the headache.  It’s impossible to get very smooth interactions without the faster camera, and with FireWire, there is no competition between devices for “attention” as there is with USB.  We also have an OptiTrack Slim:V100 from Natural Point that uses high-speed USB that we’re going to be testing.  It looks promising so far, and the fact that it does the blob tracking in hardware hopefully means that the interface will be faster.

Speaking of USB, has anyone tried installing a second USB card for their camera?  It seems like this would solve some latency issues if the camera has its’ own USB controller that it doesn’t have to share with anything else.

For our infrared illumination we are using the LED strips from environmentallights.com (also from the NUI forums).

We’ve got three ~24-inch LED strips inside the table pointing down and that seems to provide enough light to produce good blobs and to see the fiducials.  Eventually we’ll connect these to an Arduino so the computer can turn them on and off.  We may also use this same system to allow the computer to adjust the IR LED brightness depending on the ambient light level.

Because several of our applications require fiducials, we’re using the reactable software for blob and fiducial tracking.  If you don’t need to use fiducials, I think touchlib is a better option.  In our experience it has better filtering and a better calibration system.  Touchlib also seems to be a little more forgiving when tracking blobs.

We’ve been struggling with getting a nice uniform background (partially because the filters in reactable are more picky) and have found that painting the inside of the cabinet white and covering everything inside with white card stock, paper, and Coroplast (corrugated plastic) gives us a nice even background against which we can track blobs and fiducials.  We use black construction paper in areas where we get hot-spots due to reflections or ambient light.

We are using 3/8 inch plexi and a piece of vellum for the screen.  If we put anything on top of the vellum (even thin, clear plexi or polycarbonate), the fiducials get so blurry that the software can’t recognize them.  This could be remedied if we had a higher resolution camera, or two of the lower resolution cameras.  Cost-wise, two Firefly MV’s are cheaper than a 1024x768 camera with a comparable frame-rate, and two Firefly’s would have 1280x960 resolution.  As other people on this site have mentioned, it would be nice to be able to stitch together two cameras for higher resolution imaging.  We are also looking for a more permanent screen solution, and will let you know what we find.

We added an air filter and fan to the cabinet to remove heat from the projector.

Thank you to everyone who has posted their projects online.  We’ve learned a lot from the NUI community and I hope we can be as helpful in return.

We’ve got three ~24-inch LED strips inside the table pointing down

So you are using a Rear DI setup (Im assuming since you are using fiducials), but you are pointing the light down and then having it reflect back up towards the screen?  Any reason you went with this approach, as your “need” to cover everything else in your box is most likely caused due to this design?

Yes, we have a rear DI setup.

We found that when we pointed the lights directly at the screen, there was a lot of glare that was difficult to subtract out, particularly when the camera is adjusted for low-light environments (in fact, we had better results in the well-lighted environment of the exhibits shop).  Obviously our blobs were brighter when it was more directly illuminated, but they sometimes confused with hot-spots when the screen would flex.

The blob tracking software works best when there is a more uniform background, and Reactable seems particularly sensitive.  Even with filters, it’s better to start off with less brightness variation inside the cabinet.  The IR LEDs are pointing down in order to get a more even dispersion of IR across the screen.  We started by reflecting light from IR spotlights using simple white cardboard.  We got even more improvement when we moved from the concentrated IR spotlights to the more evenly spaced out LED strips.

The setup we have now seems to provide enough light for good finger sensitivity, and to read the fiducials.

THANKS FOR SHARING THIS VERY NICE PROJECT ;

what’s the screen size ?
regarding the Projector you are using , i ‘ve seen is 800*600, how about the image ?

in order to avoid that vellum problem , i have been using frosted film for windows which allow better fiducial recognition

Which software do you use to write your applications ?

Our screen is 24"x36" and the projector has a maximum resolution of 800x600.  The image I attached at the bottom of the original post shows that screen size and resolution.  There is another model of projector from BenQ, the MP522 ST that is also a short-throw projector but can handle 1024x768.  If we order another one, we’ll get that instead.

What kind of window film are you using, and where do you get it?  We’re definitely looking for something that makes an adequate projection surface yet is still transparent enough for fiducials. 

We would like to try some Vikuiti from 3M, but it’s so expensive that it’s hard to justify the expense for a test.  I’ve seen another multi-touch project that uses Vikuiti:

http://www.osram-os.com/osram_os/EN/Press/Press_Releases/Light_Emitting_Diodes/Digtal-Dash-image-projection-IR-technology-tactile-display.jsp

However, this application is much smaller than a table.  I’ve heard that the Microsoft surface uses Vikuiti for the screen material, but it’s mostly gossip, so I can’t be sure.

We’re using Flash for our applications with some VVVV in-between to pass the data from Reactable.