I recently finished an undergraduate senior capstone project in computer science in which I built a multi-touch device and software to run on it. My goals included minimal reinvention of the wheel, building on what the community made available. This is the showcase entry for my hardware.
Thanks to everyone who helped from NUIGroup and elsewhere along the way.
See attached image at bottom of post.
Hardware Quick Facts
Built into a retrofitted computer desk, using standard computer monitor, keyboard, mouse as well as a multi-touch screen
Projected rear DI
Surface: 6mm clear acrylic, about 17"x23" ("portrait" - currently projected 17” wide, but I could put a projector in sideways to get the whole surface)
Diffuser: very cheap white/off-white plastic shower curtain (from the Dollar Tree)
Illumination: four 48 IR LED illuminators from DealExtreme, powered by the computer’s 12V supply
Camera: Logitech QuickCam Pro 4000, IR filter removed, replaced with 2 layers of overexposed developed film negatives - 320x480 @ 30fps
Projector and camera on sliding drawer tethered to hinged mirror (for storage/portability)
Mirror: cheap $5 from wal-mart, not front-surface so some ghosting, but generally quite nice. (removed from frame, hinged inside chamber)
Computer: 2.4GHz Pentium 4, 1 GB RAM
Video card: BFG NVIDIA GeForce 8400GS 512mb RAM, PCI, setup for dual monitor (regular and projected MT)
Ubuntu Linux 9.04
rp-mt-scripts automatic multi-touch system - created as a part of my project, installs tbeta, standalone flash player, and pymt from version control, and allows them to be run easily)
Project Web Page for this Desk - including a video hardware tour, and a manual on setup and use of the desk that should be interesting.
Demo Video - PyMT example apps
These are some of the PyMT example apps.
Demo Video - “tactus-navigator” custom software
This is the tactus-navigator software I developed during the project, will create a new topic for it. Designed for dual-screen use.
Some quick lessons learned:
DealExtreme takes a long time to ship but the illuminators work well and are a good deal.
They don’t come with a power adapter - the center pin needs +12vdc, at least 500mA each, while the outside is the negative/ground.
They have a light sensor - I used single-side-adhesive black closed-cell foam (like for window insulation/weather stripping) to cover up the sensor under the front glass cover to permanently enable the IR. Nothing else was thick enough or opaque enough - I probably could have modified the circuit to remove it entirely but it wasn’t that big of a deal for me.
The floppy disk filter didn’t work at all for me - let through plenty of visible light and hardly any IR.
You have to put the filter where the old one was, it doesn’t work to put it outside even if it would be nice to be able to swap them out easier.
For this camera, which has a phillips chipset, setpwc was an important, easy tool to set proper frame rates. If you have a camera like this and use rp-mt-scripts, my “start tbeta” option will automatically set up camera settings first.
I tried using a sew-in interfacing material as the projection/diffusion surface, but it didn’t have as good of contrast and you could “see” the bright light from the projector through it - even though at an angle the image quality was nice, it was blinding to use.
For most of construction I didn’t have access to a good projector - I had a really really old, huge, not-always-working projector for basic things like figuring out about where to mount the mirror, etc. This wasn’t a big impediment: most of the construction can be done without a projector, and carpentry, etc. always takes way longer than you expect.
PyMT is a great MT framework and very easy to use, though the lack of documentation can be tricky and the examples aren’t always the easiest to figure out since they tend to be sophisticated - really good though especially considering how new of a framework it is. Would strongly, strongly recommend.
Innovations I Hadn’t Seen Before
Not saying nobody has done these, but I hadn’t seen them publicized - hopefully they are useful ideas:
* The projector is on a drawer and the projection/webcam mirror is hinged and tethered to that drawer, so that it reduces in size for transport and retracting the drawer for use automatically sets a correct mirror angle. This technique allows use of a small projection/sense chamber as well as the replacement of the projector with ease (it is held down to hooks on the drawer with bungee cords), yet is secure for transport (deadbolt installed on drawer). This also means that, contrary to usual practice with rear DI, the box is not completely enclosed: there is a portion open during operation due to the slide-open shelf that also permits ventilation for the projector.
* The acrylic sheet for the surface was installed into very inexpensive, plastic, false-wood-grain (also available in white) window/door edge materials. These materials were available in a number of styles including a style suitable for flat attachment and another for 90 degree angles, and came in 8’ lengths for around $2 each. They fit the 6mm acrylic used perfectly, and would also hold a stiffer projection/diffusion surface (like vellum or tracing paper) in place if desired. This was the original plan, though the improved performance of the shower curtain material and the desire for flexibility and easy replacement led to affixing the projection surface above the acrylic using double-sided tape on the plastic rails. The usual material serving this role is relatively expensive and hard-to-use aluminum or steel u-channel: this plastic material could be cut easily with a Dremel rotary tool or a hacksaw, and trimmed to fit using scissors. NUIGroup thread about this material is here
* The light sensors on the IR illuminators was disabled by the installation of a small piece of closed-cell window insulation foam, adhesive on one side, over the photoresistor after unscrewing the clear cover. Re-attaching the clear cover keeps the foam from moving away.
* Discarded projector screen material was used to line the image chamber to reflect IR emitted. It was held in place with screws and some double-sided tape. Cables for the illuminators were fed through slits cut in this lining and routed to the power source completely hidden from view within the completed chamber.
* A very cheap (Dollar Tree) off-white shower curtain was found to provide the best combination of projected image quality and touch contrast, better than a more standard thickness of shower curtain (lower contrast) and two different fabric interface materials (excessive projected light transmission, and lower contrast in the thicker material)
There are more photos on the web page, though the commentary is less focused on device builders. Each image here has a “larger” link.
Original desk - would need to have the “front” changed (so the drawers to become the MT region are on the left of the user) as well as drawer area replaced with rear DI touch system and top removed.
Power for IR illuminators: This line led into a box with a switch that I mounted below the desk, and then the four illuminators were connected to the power in parallel.
Desk stripped down - you can barely see the line drawn to show where I will cut out for the touch surface:
Overhead projector screen material cut to size for inside of the chamber: notice the large hole cut in the desk top now. The projection screen was installed just with some small screws at the top: after the cables were routed behind it I added a little double-sided tape at the bottom just to make it look even nicer.
Old printer drawer for the desk modified and reinforced to support the projector
Showing hinged mirror (mounted with the same rails used for the acrylic, tether not shown in this pic), test projector, and 3 of 4 illuminators installed in original locations. I cut slits in the projector screen below each illuminator to feed the cables through.
Cables routed behind the projector screen material to hide them from view and keep it neat - they were fed out the back and over to the power box.
Illuminators in original position (later moved the two closest to the mirror up a few inches to get them out of the view of the camera) and the camera sitting on a block of foam approximately the size of a modern projector (not my test projector) - shows the tether for the retractable drawer/mirror thing.
12V light added in parallel with the IR illuminators to show if they were switched on - ideally I’d have a nice light that was intended to be an indicator light that I could mount flatter in the desk, but I worked with what I could find quickly and cheaply.
Rail for acrylic:
Acrylic mounted, illuminators in place, and camera on mount on a nice projector, with mirror out (last view before putting on diffuser):
Any questions or more details you’d be interested to hear?
Make sure to check out the web site for more details.