Doing great work panda!

Too bad I’m not on a mac lol.

There’s all these trackers that keep forming (like Touche the other day), i’d love to talk to you sometime and maybe discuss how you are implementing some of these things and what your future plans are. Some of the new features you’re wanting to add in, are also things I’d like to add into the new tracker that’ll be a part of lux. Instead of all of us working separately just cause were doing different trackers, I think it’d help if we all collaborate so we can offer the best functionality. The one im working on, ‘tbeta’, will be multiplatform, but obviously people running on mac should have better performance with native mac code. Let me know what you think, and hopefully we can work together on all of this. Keep up the great work

There’s all these trackers that keep forming (like Touche the other day), i’d love to talk to you sometime and maybe discuss how you are implementing some of these things and what your future plans are. Some of the new features you’re wanting to add in, are also things I’d like to add into the new tracker that’ll be a part of lux. Instead of all of us working separately just cause were doing different trackers, I think it’d help if we all collaborate so we can offer the best functionality. The one im working on, ‘tbeta’, will be multiplatform, but obviously people running on mac should have better performance with native mac code. Let me know what you think, and hopefully we can work together on all of this. Keep up the great work

Hey cerupcat,

As far as implementation goes, I am happy to share with you my crazy ideas (and of course, you can always look at the code when it is all available). 

Here is my philosophical thought for the day:
As far as all the new trackers, i think it is great.  Touche looks like a great tracker, and I cant wait to see lux when it gets all done.  Since we are still kinda early days with the whole MT thing, and while it looks like we are all working separately, i think that we are actually silently collaborating more than it may seem.  I know that I have taken many ideas from the trackers that came before.  BBTouch wouldnt exist if it wast for the touchlib stuff, opentouch, and the reactivision stuff. Also, when Touche hit the scene a few days ago I went and had a look at all that great code to see how he solved the problems that I had trouble with and so on. 

I think it also helps that the ‘new trackers’ BBTouch and Lux and Touche are all very differently featured.  (just to pick on the ones that are all Mac-friendly) This way we can experiment with all of them and then the next tracker (like your tbeta) will be able to cherry pick the best ideas and build on those. 

I think that right now we are in the expansion stage of the MT stuff, right now we need to encourage all the new ideas and when there is a saturation of tracker options available, then you will start to see a coalescing of the best technologies (maybe 12 or 18 months from now i think).

Anyhow, i guess what I am saying is Yes! I agree with you! and i think it is already happening, it just doesn’t look like it on a day to day basis.

/end philosophical meandering

in any case, I would be happy to help out in any way i can.  If you want to get down to the nitty gritty i can blag on about pixel-level edge detection and region of interest mapping all day long just let me know what you want me to go on about

to answer your immediate questions (broadly):

implementation: I feel like I am running down a slightly different path than the other trackers.  I am trying to avoid ‘filters’ if at all possible (kinda). My original design goal included adding a bunch of filters into the chain, but as I was writing the detector, i changed my mind.  Now I am trying to implement the blob detection without any matrix or kernel type filters and I am focusing on just what is possible with a ‘single pixel’ of information (trying to keep the cycles down)

My thoughts are this: (and they may be totally lame, but they are mine while I personall can’t see the difference between a pixel that is value 125 and a pixel that is 126, the computer sure can.  Making the input image all filtered and sharp with nice white blobs on a black background is great to look at but it seems (to me) to be a bit of a waste of processor time when all the data you really need to track the blobs are in the source image to begin with.  Basically, after the sequence grabber delivers the bitmap data to the app, I dont want to look at any given pixel more than once.  Also there are lots of cheats you can do to accomplish some of the things that the filters are doing, but much faster.  (for instance, most everyone is applying a blur of some kind to help minimize the noise.  Instead of that, if you require blurring you should just do a downsample of the image to a smaller size (which is almost exactly the same thing as a blur, only you end up with far fewer total pixels, which will make your detector run that much faster.)

that said, I am using the background image (so technically I am doing a background subtraction filter) and the new idea will use the background image as a sort of lighting map.  darker background pixels will map to lower thresholds when comparing the foreground pixels.  so instead of taking the difference between the foreground and the background and comparing it to a global threshold (which is what is happening now) it will compare it to a local threshold (local to that one background pixel).  Theoretically this should make it track blobs in darker areas better.  It makes calibration a bit more complicated, but makes the tracking a bit more accurate.  (i implemented a version of this in my old prototype code, and it hasn’t quite made it into the office BBTouch code yet)

Of course, this might all be a really bad idea, but that is the direction I am heading with my implementation.  the core image stuff is fantastic, and it is very smart about applying all of it’s filters as a single kernel, which is super fast, but you still need to run the detector on those bits after the filters are done. So far in all my testing it takes longer to run the CI filter chain then it does to run the detector on the raw images.

My guess is that a combination of simple filters and smarter detectors is really the ultimate solution.

Anyhow I am going on and on

let me know if there is anything i can help with, or if you want a super nerdy detailed explanation

Hey All, just wanted to let anyone who was interested know that the latest revision of BBTouch is ‘out’.

as always the code is on google:
http://code.google.com/p/opentouch/

and there are binaries in the downloads section if you dont want to compile

also i wrote a quickstart guide:
http://benbritten.com/blog/bbtouch-quick-start/
it isnt quite all done (the guide) but it will get you going.

cheers!
and have fun! let me know if you find any bugs

-panda

Im sorry I dont know this but whats TUIO support mean?

Panda,

That quick start guide is really amazing and I love the new version.  I really enjoy your creative thinking.-Groovy!
I think at this point of development my mind starts to automatically think of my app’s integration in all this and I guess I start thinking about how to integrate the coordinates into my app. My question is, If you have a ROI, are blobs plotted to the coordinates of the ROI / to camera size / or from 0 - 1.  I would love it if you developed a very simple app to showcase as a guide. Something very minimal but that showed the basics of how to use the coordinate system. As you can tell its early in the morning for me too.

-ding

Hey Ding,

good question, i meant to add some stuff about the TUIO coordinates and all that stuff in the guide, but ran out of time last night so i just put up what i had

If you are using TUIO events, then the coordinates are 0..1 withing the projection mesh bounding rectangle.  (in other words, the blob coordinates get converted from the camera mesh to the projection mesh, then normalized.  this way they should all move the right direction relative to your projection and so on.

If you are planning on using Cocoa, then have a look at the BBTestAppView object, it has the code to convert from the projection mesh to the screen.  (so many coordinate spaces!) If you are wanting to start developing your own app, i suggest you just start in that view and change it to do what you want.  Soon i will have finished the distributed object blob event mechanism and at that point you could take all the objects that were previously shacked into BBTouch and re-implant them in their own app with a bit of DO to get the events from BBTouch. 

There are a few things that i realize (the morning after i ‘let the code out’ that were in the back of my mind, but never got written down and didnt make it into the release.  One of those things is to add the screen coordinates of the projection mesh into the mesh XML file so that external TUIO apps will be able to recreate a properly placed ‘active area’.

Also, i think that the meshes are great, but now the way BBTouch is handling all the various coordinate spaces is a bit busted so that will need to change in the near future as well (dont let this stop you from developing, when it does change it will require a very small amount of recoding if you code to the current standard. )

anyhow, i hope that answers your question? if not let me know!

Also, if you run into any bugs let me know as well

cheers!
panda

hey tagaiz,

briefly, TUIO is an OSC based protocol that is widely used in to communicate tangible interface information between applications. 

you can read more about it here:
http://reactable.iua.upf.edu/?tuio

and you can read all about OSC (openSoundControl) here:
http://opensoundcontrol.org/

cheers!
-panda

Hey Ding,

i should also mention that as soon as I have the distributed object code all working I will absolutely be making a demo app that can be used as a template for BBTouch client apps

Cheers!
-panda

I didn’t understand much but is TUIO basically taking the input information from your finger and telling other apps where it is? If so what does that mean in the big picture?

tagaiz,

basically that is exactly what it is.  TUIO is one of the mechanisms that is widely used to get tracking information from a tracker to an application.  There are a few other ways but TUIO is most widely adopted.  If you are developing an app and want it to be compatible with most of the trackers out there, then you want to add TUIO support. 

Cheers!
-panda

Hey Ding, (and anyone doing TUIO stuff)

So I am rethinking my decision with the TUIO coordinates, but I am not sure what is most useful for the TUIO peeps out there?

The two basic options are:

Option 1: provide the normalized coordinates like I am currently doing (normalized into the projection mesh’s bounding rectangle) this requires that your TUIO app has it’s view in the same position as the projection bounding rectangle. This means that I would need to provide some extran information (via XML file?) for where to put your TUIO app view (presumably in global screen coordinates.  I envision providing basically the frame rect, in screen coords that corresponds to where the projection frame is.)

the more i think about that the less good it seems.

Option 2:  provide blob locations normalized relative to the entire screen. this means that if you are like me, and the projection mesh is much smaller than the entire screen, I would only ever see TUIO locations in a subset of the 0..1 range (like i might only ever see events that are greater than 0.2,0.3 and less than 0.7,0.8) This way seems much better but i wonder what everyone else thinks.

I cant seem to find a really good answer in the specs.  (it seems that they always presume that you are using the entire screen) so i am lead to believe that option 2 is more in line with the specification?

cheers!
-panda

Hey panda,

0-1 from your mesh I think is right. For example, in touchlib and in tbeta, we’re doing 0-1 based on the bounding box. The bounding box is screen space, which may not be the same size as camera space. I think it’s the same thing as you mesh.

Then you send that 0-1 to the application and the application multiples it by it’s resolution. So if running an application in full screen, it’ll go from 0 - whatever the resolution of application/screen is.

The problem is, if you don’t operate your application in full screen, the blobs will be off. Usually this isn’t really an issue.

Does that help at all?

Yeah, that does help, quite a bit actually. 

In BBTouch the projection mesh is definitely not the same as the camera mesh (usually much bigger), but it also can be a subset of the screen rectangle. (for instance, on my surface, i am only using maybe 2/3 of the projection, the rest is cropped off because my surface is a long horizontal shape instead of 4:3 like the projected image)

SO i think i understand what you are saying: (correct me if i am crazy, or an idiot :-) )

I will use the projection mesh to convert the coordinates to screen-space, but then use the screen bounds to normalize the coordinates. That way when the TUIO apps are running full screen, then all the points should match up hunky-dory.  (and if it is running in a window, it will be 0..1 inside that window’s bounds) and if you are like me, and the projection mesh is much smaller than the actual projection, then that is all OK because when the apps run full screen on my surface, the coordinates will still all match up.

a concrete example: (ignoring the camera mesh for now, and assuming that the blob stuff is all working properly, just looking at my projection mesh)

my projector at full screen is 1000 x 800 lets say. (to keep the math easy)

my projection mesh rectangle is somewhere in the middle like lower left corner at 100,200 upper right corner at 900, 600. (yielding a projection mesh of 800 x 400)

I put my finger in the upper half of my projection mesh at mesh coordinate 400,300

this converts to screen coordinate:  500,500

if I normalize based on my projection mesh and projection coords i would get 0.5,0.75 (i think this is the wrong way)

but if I normalize based on the screen rect and screen coords i get: 0.5,0.625 (the right way?)

If I understand what you are saying correctly, the right one to use is the second, the screen rect.

anyway, i may be explaining it badly, but I am fairly certain i am with yah, and the second one seems like it is a more logical choice all around (which is why i think that is what you are telling me :-) ). 

(and I actually just committed an update that ‘fixes’ it in BBTouch so it is more the way that you describe, and less the ‘broken’ way i originally had it, so if i am wrong, then i will have to go back and fix it again :-) )

cheers!
-panda

Yeah, exactly! At least I think so haha.

Basically in touchlib/tbeta we have a box, where you can make sure that the mesh is fully projected on the table, just like yours. For example, on some tables, the projection is larger than the table itself and therefore you have to make the calibration mesh smaller so you can touch all the points (since some of them you won’t see since they’re won’t be projected in the table space). This bounding box is what 0 - 1 would be considered (sides of the bounding box and not the full projection).

I think you hit it spot on though. Hopefully I can get to test bbtouch soon so I can be more useful hehe =)

BTW, i’ll get you a e-mail reply shortly. My nuigroup PM box got full again which is why you couldn’t send the message.

oh, hmm.. now that you say that i think i have it backwards. that i should be using the mesh bounding box (ie the 0.5,0.75) one from above.  (or possibly i am just very confused

I guess the next question is: in your scenario, how does the TUIO app know where to draw it’s coordinates? (if your bounding box is floating somewhere in the middle of the projection)

do you provide a file telling it where the calibration box is?

thanks for the help!
-panda

brainwave!
I guess i could just go test it with one of the many TUIO test apps i have, that would be a good idea! ( durh! sometimes i am dumb! )