Hey Macdaddy,
good question, heh, it has already changed quite a bit since then so I had to go back and look to remind myself… anyway:
the right hand window is basically the ‘configure’ window, and yes the image you are seeing is (as you say) an image drawn based on what the camera sees.
the raw image looks something like this:
http://benbritten.com/blog/wp-content/uploads/2008/03/photo-43.jpg
although that one is the first prototype, but the images are very similar.
The blobs there are the ‘result’ bitmap. It is generated by the raster that subtracts the background and applies the threshold.
As far as libs, I am really only using the sequence grabber (via a highly modified version of this: http://www.skyfell.org/cocoasequencegrabber.html) and my own bitmap filter code.
I had been using some vImage stuff, but i found in the end that no matter what i did with the image beforehand, I still had to go and look at every pixel in order to do the image segmenting (ie blob finding). Since the big processor draw was the pixel-by-pixel blob search, i decided just to combine everything and do it all at once. In the end it is pretty straightforward (so far 
.
here is how it goes:
First i modified the Cocoa Sequence Grabber to generate only single color bitmaps instead of RGBA ones.
then i get a background image (i take one right as the the camera starts up, or you can take a new one if the lighting changes or you move the cam, whatever)
for every source image, and for every pixel (in the specified region of interest), check to see the value difference from the same pixel in the BG, if it is above a certain amount, it is ‘good’, go into a recursive loop finding all the pixel’s ‘good’ neighbors, repeat until there are no more pixels.
then do some basic filtering on the resultant blobs (merge any that are close enough to be the same blob, get rid of any that are too small or too big etc..) then pass those raw ‘pixelblobs’ onto the analyzer which decides if they are new, or just old blobs that have moved etc…
And that is of course, just the set of code that is working the best right now. I also started a quartzcomposer patch, and a quick blob tracking test program using QTKit/QTCapture (which is awesome, except you cant control the camera image (ie exposure etc) from QTKit. once that is possible i will absolutely abandon the SG libs in the old quicktime.
All this said, I am also looking at the vImage stuff (or core video and all that) and then simplifying my raster code. I am kinda doing it the ‘hard’ way right now because it is working, and because i needed to have a good grasp on what kind of manipulation each pixel needs before i go in throwing big filters at it (that and I think that really, at the base of things, the filters are nice and all and make it easier for me to ‘see’ the blobs, but ultimately the data is all in the original image so theoretically i should be able to get the blobs out without them (which is kinda what I am doing)
Sorry, that was kinda long and rambling, i just got up, haven’t had my caffeine yet. (and the code isnt really done yet, which is why it is all kinda vague) i hope it made some sense. I am planning on doing a blog post about it all in more detail once I have it all pinned down. feel free to ask any questions either here or email me: nui (at) benbritten.com (tho you might give me a heads up here if you do, sometimes my junkmail filter can be overly aggressive
-panda
ps: i would be more than happy to send you the code I have so far, but it is in a gigantic mess, and is in constant flux at the moment. If you give me a week or two more ( maybe three i am shooting for an early beta that you could just install (or compile yourself, or modify or whatever)
