trouble isolating fingertips from hand/arm
Posted: 25 January 2008 06:06 PM   [ Ignore ]
Rank
Joined  2008-01-25
Total Posts:  2
New Member

Hello, This is my first time in a forum so i hope this thread is located correctly. I have been working on a DI multitouch interface for a while now but for learning purposes I made some C# .NET classes to capture the image from a usb webcam, process the image, track blobs and dispatch a Touch, TouchMove and UnTouch events. The software part is working as expected, I can control the binarizing threshold, blur, frame rate, minimum and maximum blob area, blob aspect ratio, image invert(to detect shadows when working with external IR light). In the attached Images you can see each blob has an Id(red number), Area (top blue number), XY coordinates (Bottom blue numbers). For hardware I am running an hp dv8000 laptop, amd64 2,0 Ghz 1Gb ram, creative usb cam at 320x240, 6 IR leds and a 5mm frosted glass. The problem I have is that the part of the hand that is touching the glass is almost the same brightness as the parts that are near but not touching so the palm of the hand and/or the knucles are being interpreted as a touch. I tried implementing an algorithm to detect the fingertip based on the shape of the hand but that implies more processing power which I dont want to sacrifice. With this system the cpu goes up to a 27-34% running at 320x240 10 fps. I havent tried touchlib so I would like to ask you guys if this is decent preformance or if I should switch to touchlib to get better performance.

So basically the questions are:
1. How to increase the contrast of the objects touching the glass vs objects not touching the glass.
2. How does my cpu load compare to similar machines running touchlib.

Thanks!

Image Attachments
mtJk.JPG
Profile
 
 
Posted: 26 January 2008 07:08 PM   [ Ignore ]   [ # 1 ]
Avatar
Rank
Joined  2007-09-25
Total Posts:  84
New Member

Hi, it would be cool to check out your code if you think that’s okay. I think many of this group can learn something from it. And maybe we can help you make it better/faster. I don’t realle know how much touchlib is asking, but I can say it will be lots. Because of the constant data input from the camera, and processing of the touches. Count in another performance lose when showing some nice graphics and you need a high-end pc currently available to make it work smooth…

to change the contrast I can’t think of something else then a constrast ratio matrix to put over the image…

Oh and I can tell you. I have an Athlon FX55, with 2 Gb of DDR2 ram and a 256 Mb ATI RADEON that it doesn’t run that smooth, but that was in a bad testmode and with lots of applications on…

 Signature 

http://www.michaelvanderheeren.be/

Profile
 
 
Posted: 26 January 2008 07:14 PM   [ Ignore ]   [ # 2 ]
Avatar
Rank
Joined  2007-09-25
Total Posts:  84
New Member

Something extra, I don’t know how your program is coded, but I saw you’re using C#. That’s not a good idea, it has the same cost as using VB.net. They are not low level programming language and require some convertion to work with the system. That all happens using .NET framework. The best thing is to create a native C++ library to keep track of your touches, and import that into a .net project. That way you can reduce the load of capturing your touches…

 Signature 

http://www.michaelvanderheeren.be/

Profile
 
 
Posted: 27 January 2008 05:07 AM   [ Ignore ]   [ # 3 ]
Avatar
RankRank
Joined  2007-04-03
Total Posts:  241
Moderator

See like the frosted glass doesnt creates enough contrast differences, have you tried another glass material with a better diffuser?

 Signature 

My multitouch blog: http://www.multigesture.net
Howto: Compile touchlib on windows XP/Vista
Howto: Compile touchlib on Ubuntu Linux
Downloads: Touchlib SVN builds

Profile
 
 
Posted: 27 January 2008 06:55 AM   [ Ignore ]   [ # 4 ]
RankRankRankRank
Joined  2007-07-14
Total Posts:  819
Elite

Can it be that you are not removing a background?

 Signature 

HID Multitouch driver for Windows 7 http://multitouchvista.codeplex.com/

Profile
 
 
Posted: 27 January 2008 07:11 AM   [ Ignore ]   [ # 5 ]
Rank
Joined  2007-10-22
Total Posts:  92
New Member
SoniX - 26 January 2008 07:14 PM

using C#. That’s not a good idea [...] The best thing is to create a native C++ library [...] That way you can reduce the load of capturing your touches…

I highly doubt that it is a valid statement in general. I can really only speak for Java, but as C# is somewhat similar to Java (speaking about virtual machines and managed code and such things), performance should be similar.

It has long ago (3 years or so) been shown that performance-wise, Java is pratically identical to C++, with some advantages here or there for one or the other. C# probably performs as well, too. Performance is really mainly about algorithms and good code, not programming languages. If your algorithm avoids work load, then it will perform well. If you’re scanning an image for blobs, performance depends on your scanning algorithm. There’re a few ways to avoid scanning all 640x480 or whatever pixels. Take only every second pixel (for 99% of the people in this community, that should by far be enough), and you only have to process 1/4 of the pixels. Increase the value and take every third, and you’ll even do much better.

My computer is by far not high-end, just a normal dual core Athlon 2600 (2x2400MHz), and with 640x480 and the method described above, the processor is at 5-10% at 30fps - although this is more of a test case, I don’t have had the chance to test it with a real multitouch program running, but it should be similar. Running on a 6-year-old Athlon XP 1700+ (1366MHz), this time in a real working environment, it works very stable at 15 fps, taking about 50% of the processor. Given that the USB cameras suck performance-wise, it should even be better when I finally get a Firewire cam in my hands.

Profile
 
 
Posted: 27 January 2008 08:54 AM   [ Ignore ]   [ # 6 ]
RankRank
Joined  2007-06-06
Total Posts:  139
Member

Hey Juank,
Your diffuser material will make all the difference in this situation.  I’ve used a thin sheet of vellum paper and had to do more aggressive filtering to get rid of my palm and other hovering fingers.  When I starting using rosco grey for DI though (which takes way more light) the thickness helped to filter out anything other than my fingertips.  So… try more diffuser materials smile

 Signature 

- Miketavius

Profile
 
 
Posted: 27 January 2008 09:30 AM   [ Ignore ]   [ # 7 ]
Avatar
Rank
Joined  2007-09-25
Total Posts:  84
New Member

@ElkMonster: sure you’re right about simple algorithms, but ever tried to calculate circle intersections in about 10000 circles. You’ll see that Java will need good code, and I mean such code that it only works for that project, to make it out as good as c++. And that not what we need in the industry. We need good code that is fast to use. And off-course Java has also a lot of advantages, like cross platform and the fact that it’s a lot easier than c++ but more limited due to not having pointers. I did the circle intersection thing for an university work. Thrust me, I wrote good code, code that can be reused for other things, and then java came out as a middle class language if we talk about miliseconds and very big problems. I’m sute this was due to the jit-compiler…

But what i mean in fact is not the overhead of the normal algorithms, that will be small for normal problems. The thing is that you need to capture video, and in most cases we will use opengl or directx for that. But most of those are designed in and for languages like c++. Off course they can be used in vb or c#. But then again there is some extra code between them to make it work. And it is that code that creates the overhead. A good example was they BAD way system.drawing was implemented in .net… It was a lot slower than the hard way to draw dots, lines,…

And let us also agree that coding is a choice, and if you’re either a java or c adict, both have advantages and disadvantages…

 Signature 

http://www.michaelvanderheeren.be/

Profile
 
 
Posted: 27 January 2008 11:26 AM   [ Ignore ]   [ # 8 ]
Rank
Joined  2008-01-25
Total Posts:  2
New Member

Hey Guys, thanks for your feedback! Im gonna try out with other surfaces/diffusers, increasing the opacity and probably Rosco (If I can get my hands on it in my country), an update the code to process every other pixel and measure the performace increase. Ill post back my results. 

Thanks!

Profile
 
 
Posted: 28 January 2008 09:35 AM   [ Ignore ]   [ # 9 ]
Rank
Joined  2007-10-22
Total Posts:  92
New Member

@SoniX: I’d like to discuss this further, but maybe we should open our own thread and not spam this one. wink Also, my time is very limited currently, so maybe in two weeks or so… smile

Just the fact I wanted to point out again: Those languages are fast enough to do “realtime” (as far as this is possible, considering camera delay etc.) blob tracking with it. Other than that, there might be very noticable differences (coming to OpenGL etc., which but is something more specific). I don’t know whether pixel scanning is normally done through OpenGL or DirectX - actually, it should be done working with bare bytes, that is, the camera driver copies the image data to memory, and you read this data directly. No OpenGL or DX necessary.
Err, I’m writing too much again…—anyway, as said above: to be continued some other day… wink

Profile