New Method for FTIR - Cameras connected to LEDs
Posted: 12 March 2010 05:48 PM   [ Ignore ]
Avatar
RankRankRank
Joined  2009-11-09
Total Posts:  489
Sr. Member

This might be very intersting for the NUI-Community and MT development.
Today i received a master-thesis from a friend who graduated two days ago.

Here is a summary:

“Abstract
Even though multi-touch screens based on frustrated total internal reflection
(FTIR) principle have gained significant popularity over the last years their
overall performance is often limited by design. In particular, we identified robustness
towards ambient light, high-speed tracking and precise calibration as key
issues in many systems. In this thesis we propose a set of software and hardware
extensions for a custom-built FTIR screen that improve on these aspects by reducing
the impact of ambient light, optimizing tracking performance and performing
an automated calibration at high precision. Comparisons with existing approaches
reveal great optimization potential: Our system performs well under intense and
rapidly changing lighting conditions, its tracking runtime is up to 30x faster than
Touchlib and calibration achieves sub-pixel resolution. The results are general
enough to apply equally well to other camera-based technologies and might influence
the next generation of multitouch processing software.

Despite its research history, only in recent years with the introduction of the
FTIR1 approach the number of researchers and enthusiasts realizing the potential
of the technology grew significantly. Several companies [Mic07, Per07, IB07]
have also developed products based on FTIR or closely related technologies. Even
a small selection of libraries and SDKs dedicated to multitouch-specific processing
tasks are openly available and the acceptance of the technology is increasing.
Still, the used software and hardware components are in a very early stage.
Over the course of a year we have seen many different system configurations, but
were mostly disappointed of the overall performance. First, either tracking accuracy
or latency or both altogether were not satisfying enough to use the touch
screen as a primary human computer interface (HCI). Second, the lack of robustness
towards ambient light limit their applicability. Third, systems exhibit errorprone
calibration methods based on tedious manual procedures. We identified
these three aspects as key issues that deserve a more thorough discussion.
This thesis aims to present ideas from established research areas, translate
them to this domain and discuss how they apply and perform.”

Basically we connected the cameras with LED frame...simple explanation smile

cheerz
goat

 Signature 

My DSI Project:

DSI Table
Proj.Surface: 7D006 on top
2x PS3 Camera from Peau with 850BF & 3.6mm lense
Endlightend XXL (10mm)with 850nm LED strip

Projector: SANYO DWL100 ~ 950€

PC:
Core i5, 8GB Ram, HD5850

Profile
 
 
Posted: 12 March 2010 06:22 PM   [ Ignore ]   [ # 1 ]
Avatar
RankRankRankRank
Joined  2006-11-09
Total Posts:  1499
Administrator

Could you please attach the pdf?

Cheers Chris

 Signature 

~

Profile
 
 
Posted: 12 March 2010 06:57 PM   [ Ignore ]   [ # 2 ]
Avatar
RankRankRank
Joined  2009-11-09
Total Posts:  489
Sr. Member

Im sorry i cant publish the complete PDF - These are parts which were already published:

Chapter 3 - “Tracking Performance:

Among the most crucial requirement of a HCI device is its ability to quickly respond
to user input, because if the latency between user input and its processing
result is too high, the device feels unattractively slow and lagging. At the same
time, the device should not react randomly, but exactly in the particular way the
user envisioned it. Thus, the goal is to achieve low latency and low error rate
simultaneously. The tracking performance of an FTIR screen plays an important
role in achieving these goals. This chapter discusses a system design that is able
to achieve fast and robust multitouch tracking.
In the following sections we will give a problem definition, provide a solution
using the forward/reverse auction algorithm, evaluate the system and compare
runtime and accuracy with a widely-used open source multitouch library.
..
.

3.1 Problem Definition
Tracking refers to the process of finding correspondences between object features
of two or more images that belong to the same object. Object tracking is an established
technique used in many fields such as in motion analysis and compression.
Widely-used algorithms include the KLT feature tracker [LK81] and optical flow
estimation [HS81].
But tracking FTIR signals deviates from these classic real-world problems. A
signal in an FTIR setup is a small blob whose shape and intensity can rapidly
change. The signal data is actually more similar to radar imagery than it is to
a real-world scene and each object is reduced to a single coordinates. But also
algorithms for multiple target tracking, i.e., radar tracking [Bla86] using Kalman
filter [Kai68], are not optimal, because velocity and direction of an FTIR signal
is, unlike a plane, subject to sudden unpredictable change.
20 Chapter 3: Tracking Performance
At the same time, a high number of trackable objects can be present simultaneously
as large-scale FTIR setups are possible. Therefore, a tracking algorithm
should not considerably slow down the system for any reasonable number
of touches.
Hence, a tracking method for FTIR signals should:
• find robust correspondences between coordinates in two frames.
• maintain low computational complexity, even for larger data sets.
The following section discusses the forward/reverse auction algorithm as a
possible choice for a tracking algorithm.”

...
..

3.5 Conclusion
We presented a design for FTIR signal tracking based on the forward/reverse auction
algorithm. The system was shown to achieve low latency and low error rates
even for larger tracking problems. Alternative approaches based on established
tracking algorithms such as KLT feature tracker and optical flow are either not
compatible with FTIR signal properties or computationally prohibitive. We reduce
raw images to trackable points and showed that point tracking can have low
computational complexity while a robust tracking with an error rate around 2%
can be achieved.
We compared our approach to a widely-used open source multitouch library.
The forward/reverse auction algorithm performs between 5x to 30x better in runtime
and equally for small and even for better larger problem sizes. In a brief
analysis of the Touchlib library we discussed its limitations.”

Image Attachments
touch.JPGtouch1.JPG
 Signature 

My DSI Project:

DSI Table
Proj.Surface: 7D006 on top
2x PS3 Camera from Peau with 850BF & 3.6mm lense
Endlightend XXL (10mm)with 850nm LED strip

Projector: SANYO DWL100 ~ 950€

PC:
Core i5, 8GB Ram, HD5850

Profile
 
 
Posted: 25 March 2010 04:25 PM   [ Ignore ]   [ # 3 ]
Avatar
RankRankRank
Joined  2009-11-09
Total Posts:  489
Sr. Member

@all

so what do you think ?

“Our system performs well under intense and
rapidly changing lighting conditions, its tracking runtime is up to 30x faster than
Touchlib and calibration achieves sub-pixel resolution. The results are general
enough to apply equally well to other camera-based technologies and might influence
the next generation of multitouch processing software.”

last week i saw the table in action. it is by far the most responsive system… even faster than the PQLabs IR-Frame i saw at CEBIT. he also presents a fast&easy-30-sec-autocalibration - it works perfect!

isnt it somehow possible to “adapt” this method to CCV?

@Christian & AlexP and other developers?

 Signature 

My DSI Project:

DSI Table
Proj.Surface: 7D006 on top
2x PS3 Camera from Peau with 850BF & 3.6mm lense
Endlightend XXL (10mm)with 850nm LED strip

Projector: SANYO DWL100 ~ 950€

PC:
Core i5, 8GB Ram, HD5850

Profile
 
 
Posted: 26 March 2010 02:28 AM   [ Ignore ]   [ # 4 ]
RankRankRankRank
Joined  2008-11-22
Total Posts:  529
Elite

hi goat
i think it is possible when he developers see the algorithm. did you ask your friend to share this beautiful algorithm?

 Signature 

my 19” LCD setup ----> turned into new look
my 60” FTIR setup (old video)

Profile
 
 
Posted: 27 March 2010 01:49 PM   [ Ignore ]   [ # 5 ]
Avatar
RankRankRank
Joined  2009-11-09
Total Posts:  489
Sr. Member

.... zipper

 Signature 

My DSI Project:

DSI Table
Proj.Surface: 7D006 on top
2x PS3 Camera from Peau with 850BF & 3.6mm lense
Endlightend XXL (10mm)with 850nm LED strip

Projector: SANYO DWL100 ~ 950€

PC:
Core i5, 8GB Ram, HD5850

Profile
 
 
Posted: 27 March 2010 04:33 PM   [ Ignore ]   [ # 6 ]
Avatar
RankRankRank
Joined  2008-05-08
Total Posts:  342
Sr. Member
the_goat - 27 March 2010 01:49 PM

here is the extracted chapter that explains the used algorithm.

i would love to get some feedback on this from christian, alexp and other developers.

Could you please post the whole thesis. The Chapter 2 does not explain the “forward/reverse auction algorithm” used in your project. As far as the LED strobing and the background subtraction outlined in chapter 2.4, this is doable but requires some hardware to be built in addition to code modifications to the CCV.

 Signature 

Computing is not about computers any more.  It is about living!

Code Laboratories
CL Studio Live - Video like never before
CL Eye Platform SDK

Profile
 
 
Posted: 27 March 2010 06:52 PM   [ Ignore ]   [ # 7 ]
RankRank
Joined  2010-02-06
Total Posts:  173
Member

Wow,

This is some really super impressive stuff. So the question is, with this be made public either by way of open source with a breakdown of hows its done or commercially. Ether way I am sure there are many folks here that would love to see this in action. I for one am always looking for alternative ways of doing things.

Killer work

Sam

 Signature 

Screwfunk
TouchMi inc. - 2 meter Prototype RearDI Bar

Profile
 
 
Posted: 04 May 2010 04:23 PM   [ Ignore ]   [ # 8 ]
Rank
Joined  2010-05-04
Total Posts:  1
New Member

Hi,

LED strobing and syncing of camera captures/shutters are widely used in the motion capture areas for film and games. Typically cameras are synced via some TTL signal to a master clock such that multiple camera setups have correlating 2D data to process. This is very important for 3D reconstruction techniques.

As for the auction algorithm. Info can be found here: http://wapedia.mobi/en/Auction_algorithm as well as here: http://en.wikipedia.org/wiki/Auction_algorithm.
It is mainly a bipartite graph matching technique of which the ‘Hungarian’ method (http://en.wikipedia.org/wiki/Hungarian_algorithm) is the most well known. The time complexity of the Hungarian algorithm is O(n^4), although there are heuristic adaptations to make it run in O(n^3). Here is a nice algorithmic explanation of the Hungarian algorithm: http://www.topcoder.com/tc?module=Static&d1=tutorials&d2=hungarianAlgorithm

ciao,
Sean.
PS: this is my first post on this forum, and yes, I’ll be building my own multi-touch surface able to track fingers and fiducials (the hardware is just the start - I’m more interested in the software side of things....)

Profile
 
 
Posted: 25 October 2010 08:39 AM   [ Ignore ]   [ # 9 ]
Avatar
RankRankRank
Joined  2009-11-09
Total Posts:  489
Sr. Member

unfortunately i didnt have the time to do some more research on this subject even though it is really interesting because it could lead to very stable (terms of ambient light) mt-setups. but does any of you know a working setup with “pulsed leds” or similar technology?

 Signature 

My DSI Project:

DSI Table
Proj.Surface: 7D006 on top
2x PS3 Camera from Peau with 850BF & 3.6mm lense
Endlightend XXL (10mm)with 850nm LED strip

Projector: SANYO DWL100 ~ 950€

PC:
Core i5, 8GB Ram, HD5850

Profile
 
 
Posted: 25 October 2010 09:08 PM   [ Ignore ]   [ # 10 ]
Avatar
Rank
Joined  2008-07-16
Total Posts:  20
New Member

Hey the_goat, this is from a Masters thesis right? You should be able to publish the entire PDF as it becomes public domain after graduation (at least it does here in Australia) for general reading. It would be interesting to fill in the blanks.

Profile
 
 
   
 
 
‹‹ PS3Eye Query      Reinvent the wheel! ››