NUI Group » OS X Multitouch?

Posted: 29 July 2008 04:01 AM

[ Ignore ] [ # 61 ]

New Member

Total Posts: 5

Joined 2008-07-28

Hi Panda!

Thanks a lot for your help. Actually I didn’t manage to cache the ciContext (I have a crash when using it at second call, very strange...). Anyway I used the code you proposed and it works perfectly! I still have some problem with the color conversion but it should be easy to fix since I can “see” the image data now

Have a nice day and thanks again for your detailled answer.

Franck

Profile

Posted: 29 July 2008 04:19 AM

[ Ignore ] [ # 62 ]

New Member

Total Posts: 83

Joined 2008-03-23

hey franck,

no worries! i am just glad that others wont have to go through the long frustrating process that i did to find all that stuff out

anyhow, you should be able to simple log out the bitrep to find out the colorspace:

ala:

NSLog(@"bitmap thinger: %@”,bitRep);

it will spit out a bunch of info about the rep, including the color space. another way is to look at the first four bytes (as unsigned chars)
if you are coming from a video buffer, then the alpha will almost certainly be 0xff (255). so have a look at the first four bytes, if they look like:

NSLog(@"%d %d %d %d”,data[0],data[1],data[2]data[3]); // presumes you have cast your data as an unsigned char *
you wil; get:
255 37 26 192 (or some other rgb values)
then it is ARGB
and if it looks like
37 26 192 255 then it is RGBA (natch!)

I am fairly confident tho, that if you are using the code from before that it will be ARGB (but i have been wrong before

cheers!
-panda

Signature

Profile

Posted: 30 July 2008 04:02 AM

[ Ignore ] [ # 63 ]

New Member

Total Posts: 5

Joined 2008-07-28

Hello again Panda,

Thanks for this bitmap thinger, it shows that I receive a RGBA buffer. Anyway I still have an issue with colorspaces: I presume that the initWithCIImage did not care about it and then depending on the video source I have correct or incorrect colors in my final buffer. I’m trying to investigate…

Franck

Profile

Posted: 30 July 2008 04:26 AM

[ Ignore ] [ # 64 ]

New Member

Total Posts: 83

Joined 2008-03-23

Franck,

RGBA huh? well i was so totally wrong (wont be the last time

Anyhow, the initFromCIImage should convert the CIImage color properly to the RGBA. But i guess from there i dont know what space you need? If you need to convert to something esoteric like HSB or YUV then you should go back to the code ding posted as you can specify the destination colorspace.

if you need something like RGB (no A) then you will probably have to do the conversion yourself (which should be a quick for loop through the RGBA stuff and build a new buffer)

if you need something like planar_8888 then you should look into the vImage stuff.

Cheers!
-panda

Signature

Profile

Posted: 30 July 2008 04:32 AM

[ Ignore ] [ # 65 ]

New Member

Total Posts: 5

Joined 2008-07-28

Panda,

Actually I need something like RGB so a simple loop is enough. Anyway what I observe is that blue objects on my input video turn to pink in my final buffer. It does not seem to be an inversion of R, G or B…

Franck

Profile

Posted: 30 July 2008 04:37 AM

[ Ignore ] [ # 66 ]

New Member

Total Posts: 83

Joined 2008-03-23

pink eh? funky!

how are you grabbing frames from the video? that also might be the source of the color conversion problems.

Groovy!
-b

Signature

Profile

ding

Posted: 01 August 2008 08:51 PM

[ Ignore ] [ # 67 ]

New Member

Total Posts: 59

Joined 2007-07-09

I havent had internets in like a whole week and I finally found a restaurant that has free wifi so here goes very quicklike before my dinner gets here:

+ (IplImage*) iplImageFromCVPixelBufferRef:(CVPixelBufferRef) image {
    // Note, we're assuming here that the data is coming in format k32ARGBPixelFormat
    
    CvSize size = cvSize((int) CVPixelBufferGetWidth(image), (int) CVPixelBufferGetHeight(image));
    
    IplImage* result = cvCreateImage(size, IPL_DEPTH_8U, 3);

    CVPixelBufferLockBaseAddress(image, 0);
    char* inData    = CVPixelBufferGetBaseAddress(image);
    int inRowBytes  = CVPixelBufferGetBytesPerRow(image);
    int inPixOffset = 4;
    
    char* outData   = result->imageData;
    int outRowBytes = result->widthStep;
    int outPixOffset = 3;
    
    int x, y;
    for (y = 0; y < size.height; ++y) {
        int inOffset  = y*inRowBytes;
        int outOffset = y*outRowBytes;
        for (x = 0; x < size.width; ++x) {
            int xOutIdx = outOffset + x*outPixOffset;
            int xInIdx  =  inOffset + x*inPixOffset;
            outData[xOutIdx]     = inData[xInIdx + 3 /* blue */];
            outData[xOutIdx + 1] = inData[xInIdx + 2 /* green */];
            outData[xOutIdx + 2] = inData[xInIdx + 1 /* red */];
        }
    }
    
    CVPixelBufferUnlockBaseAddress(image, 0);
    return result;
}


+ (IplImage*) iplImageFromCIImage:(CIImage*) image
{
    CvSize size = cvSize((int) image.extent.size.width, (int) image.extent.size.height);
    
    IplImage* result      = cvCreateImage(size, IPL_DEPTH_8U, 4);
    IplImage* finalResult = cvCreateImage(size, IPL_DEPTH_8U, 3);
    
    CGColorSpaceRef colorSpace = CGColorSpaceCreateWithName(kCGColorSpaceGenericRGB);
    CGContextRef context = CGBitmapContextCreate (NULL,
                                     size.width,
                                     size.height,
                                     8,      // bits per component
                                     result->widthStep,
                                     colorSpace,
                                     kCGImageAlphaPremultipliedLast);
    if (context == NULL) {
        NSLog(@"Context not created!");
        return NULL;
    }

    CIContext* ciContext = [CIContext contextWithCGContext:context options:[NSDictionary dictionaryWithObject: (NSString*) kCGColorSpaceGenericRGB forKey:  kCIContextOutputColorSpace]];
    
    CGRect rect = CGRectMake(0,0, size.width, size.height);

    // Expensive...
    [ciContext drawImage:image inRect:rect fromRect: rect];
    
    // Expensive...
    [ciContext render:image 
             toBitmap:result->imageData
             rowBytes:result->widthStep
               bounds:rect 
               format:kCIFormatARGB8 
           colorSpace:NULL];

    // Reorder ARGB to RGBA
    int i;
    for (i = 0; i < result->imageSize; i += 4) {
        result->imageData[i]   = result->imageData[i+1];
        result->imageData[i+1] = result->imageData[i+2];
        result->imageData[i+2] = result->imageData[i+3];
    }
    
    // Change to the standard OpenCV color ordering.
    cvCvtColor(result, finalResult, CV_RGBA2BGR);
    
    // Cleanup
    CGColorSpaceRelease( colorSpace );
    CGContextRelease( context );
    cvReleaseImage(&result);
    
    return finalResult;
}



+ (CIImage*) ciImageFromIplImage:(IplImage*) image
{
    IplImage* imageClone = cvCreateImage(cvSize(image->width, image->height), IPL_DEPTH_8U, 4);
    cvCvtColor(image, imageClone, CV_BGR2RGBA);
    
    char*        data = imageClone->imageData;
    CGSize       size = CGSizeMake((CGFloat) image->width, (CGFloat) image->height);
    
    // Reorder RGBA to ARGB
    int i;
    for (i = 0; i < imageClone->imageSize; i += 4) {
        data[i+3] = data[i+2];
        data[i+2] = data[i+1];
        data[i+1] = data[i];
        data[i]   = 0xFF;
    }
    
    NSData*  dataWrap = [NSData dataWithBytes:data length:imageClone->imageSize];
    
    CIImage* ciImage = [CIImage imageWithBitmapData:dataWrap
                                        bytesPerRow:imageClone->widthStep 
                                               size:size
                                             format:kCIFormatARGB8
                                         colorSpace:CGColorSpaceCreateWithName(kCGColorSpaceGenericRGB)];
    
    cvReleaseImage(&imageClone);
    return ciImage;
}

I would use the CVImageBufferRef directly from the:

- (void) captureOutput: (QTCaptureOutput*) captureOutput
   didOutputVideoFrame: (CVImageBufferRef) videoFrame 
      withSampleBuffer: (QTSampleBuffer*) sampleBuffer
        fromConnection: (QTCaptureConnection*) connection

That is the fastest method as far as speed is concerned the ciimage conversions take a shitload of time.

ding

Profile

Posted: 04 August 2008 09:21 AM

[ Ignore ] [ # 68 ]

New Member

Total Posts: 5

Joined 2008-07-28

Ding & Panda,

Thanks for your answers. I finally used a mix of both your approaches and it seems ok, except for the blue turning to pink, which I read could happen due to some transformations between RGB and YUV color spaces (saturations). Now I must track these potential transformations and fix them if possible.

See you and thanks again!
Franck

Profile

http://www.youtube.com/watch?v=g0ychb_yZAw

Posted: 04 August 2008 11:37 AM

[ Ignore ] [ # 69 ]

New Member

Total Posts: 11

Joined 2008-06-09

Panda,

1st. Thanks for BBTouch.

2nd. Sorry about the Demo Text....

3rd. Havin’ a problem with BBTouch. Once I start it it won’t quit proper. Nor will it respond to force quit. (Maybe I don’t know the process that is holding it up a la Activity Monitor...)

4th: The error you see at the beginning of the video is code 600

What do you think??

uptodate 10.5

Note: At one time it did quit proper....

Profile

Posted: 06 August 2008 11:03 AM

[ Ignore ] [ # 70 ]

New Member

Total Posts: 11

Joined 2008-06-09

Sorry if this was the wrong spot to go posting this…

But even after deleting the .plist and getting a fresh opentouch download and rebuilding BBTouch it remains unresponsive to force quit and refuses to completely quit.

Is there something that I am missing?

Many Thanks!

Profile

thescreamingdrills

Posted: 06 August 2008 12:31 PM

[ Ignore ] [ # 71 ]

New Member

Total Posts: 84

Joined 2008-05-17

samalotriviera: I haven’t had that problem, can you give us any more details?

Signature

Nuigroup wiki

Profile

Posted: 06 August 2008 02:35 PM

[ Ignore ] [ # 72 ]

New Member

Total Posts: 11

Joined 2008-06-09

I can try.

BBTouch operates perfectly until you file>quit.

Then it just sits in the dock, no obvious processes remain running....
You can’t quit it, nor can you force quit it (from the dock nor Activity Monitor *see prior line)

I’ve destroyed all files associated with BBTouch, the program and even trashed the whole OpenTouch Download.

SVN’d a new copy rebuilt, restarted, retried, etc.

Run it one time, use it as long as you like but if you quit it, it just stays there. The window goes away, the menu bar goes away, it looks like you’ve gone back to the Finder. (just as in that little vid I posted)

I noticed today, when I ran it and tried to quit, same as usual, but in my frustration I said: “@#($&%^ it, I’m just shutting down.”

Mail was running and maybe iTunes or something else. BBTouch aborted the shut down! But when I quit all the other running programs it did not abort the shut down!? Interesting no?

And as i said in my previous post it, at one time, did quit and run perfectly fine. (something is hanging around somewhere in the background that continues running I guess??) I thought it might be the TUIO packets but I made sure to stop that before quitting numerous times and still it persists…

If you try to dbl-click the app again after you’ve quit it, it presents you with an error “-600”, but if you duplicate the app and click the copy it runs the copy and closes the original. When you quit the dupe, same thing. It never quits fully.

I’m sure that duping it was not such a hot idea and may have @#$@#$ something worse, but I was like “What the heck is going on here!?”

That’s all I know....hope that this kind of helps…

Any suggestions for gathering more intel are welcome (I’m not a programmer or anything, but I’m savvy enough to figure out most standard UI stuff so if you want me to look in the console, Xcode, or anything somewhere in the background, etc. , just let me know and I can do my best to ascertain the information.)

Many thanks to all!!! This MT stuff is the Cat’s Meow!

Profile

Posted: 14 August 2008 01:02 AM

[ Ignore ] [ # 73 ]

New Member

Total Posts: 83

Joined 2008-03-23

hey samalotriviera,
sorry it took me so long to post, i hadn’t noticed this thread was active again if you are ever really frustrated, feel free to post directly to my blog (that way i get a notification) or email me directly: (i do get notifications on these posts, but not always, i think maybe if I am the author then i always get them and if not then i get them if i am the last poster? i don’t know) I definitely want to have the problems and solutions here on this forum (so it is easy for others to find it) but if I am conspicuously absent for many days, then let me know

anyhoo.. about your problem..

that is a strange one and I haven’t run into it yet. It probably has to do something with the sequence grabber not shutting down properly (or there is a memory leak somewhere, or possible something else entirely)

next time you run into this problem open your console app (will be in your Applications->Utilities unless you have moved it somewhere else) and see if there are any logs with BBtouch as the source. there may be some clues there.
it will look something like:

2008-08-14 16:00:32.644 BBTouch[1763:10b] SGNewChannel() returned -9405
2008-08-14 16:00:32.647 BBTouch[1763:10b] could not start camera

Cheers!
-panda

edit: if you are running this from xcode, then the console is the xcode console, you dont need to get the console.app out

Signature

Profile

Posted: 14 August 2008 01:11 AM

[ Ignore ] [ # 74 ]

New Member

Total Posts: 83

Joined 2008-03-23

samalotriviera, one last thing:

what are you running on? ie mac model, (you mentioned 10.5), what kind of camera etc..

I am sure we can fix it

cheers!
-panda

Signature

Profile