Adding videowrapper to touchlib
Posted: 10 January 2007 01:01 PM   [ Ignore ]
RankRankRankRank
Joined  2007-01-08
Total Posts:  1008
Member

Hello guys,

I am working on adding video wrapper to touchlib at the moment. Everything
is compiling fine and it was very easy to figure it out (kudos for that),
but I’m trying to use the directshow filter and it doesn’t seem to be
working for me. I get ‘connection to video device failed’ for any of the vc:
connect strings in the example app. Any clue why it’s not working for me?

-Dave

----------------

Make sure you are using the right device number; if you turn on
debugging in the videowrapper directshow dll, it should print out all
the modes that the selected device supports.  You could modify it
easily to print out a list of all devices, first (look at the
printmodes function further down).

---------------

Ok, thanks for the info - I will take a look at that today. BTW, I posted
some videos of my screen at http://www.whitenoiseaudio.com/blog/ . Still
lots of room for improvement, but all the pieces are there.

-Dave

--------------

Cool stuff.  What are your “camera” plans, to increase frame rate?  I
wonder if 30 fps is enough, or if we will need 60 or higher (I’d love
it if we could do 640x480 @ 200 fps in BW, using a DragonFly Express).

Also, of course, is the question of resolution ... perhaps a 1024x768
30fps BW camera (like the DragonFly2) would be better than a lower
res, fast one?

Of course, those are $1000 cameras.  Hopefully, the 640x480 @ 60fps
FireflyMV we have will do, or the 640x480 @ 30fps BW DragonFly.

I suspect we’ll try the BW dragonfly first.

------------

Right now I have a Philips CCD webcam (only $79 wink that can do 640x480 @ 60
fps - the reason it’s running at 320x240 @ 15 fps is because our current
directshow driver is incomplete. I will probably stick with that thing for
now and upgrade if/when I have the money. The DragonFly2 does look good and
I believe it’s the same camera Han uses. Anything faster than 60 fps is
probably overkill.

Re: resolution - I’m wondering if it’s possible to get sub-pixel accuracy in
software by using interpolation to find the center of a blob.. Haven’t
experimented with it yet though.

-Dave

------------

Ya, that’s probably ok.

> Re: resolution - I’m wondering if it’s possible to get sub-pixel
> accuracy in
> software by using interpolation to find the center of a blob.. Haven’t
> experimented with it yet though.

Definitely!  If you get a blob, and find the weighted center (where
the weight takes the grey values at the edge into account) you should
get a subpixel center.  OpenCV should be able to do this for you, I
think.

--------------

I have video wrapper added to touchlib now. I had to make a change to
the codevis Vidcap library in order to get it to return additional
modes with higher frame rates so that I can use my camera with it. By
default it only gives me 640x480 @ 10 frames per second. With the
change in place, I can now do 640x480 @ ~50 fps. No changes were
necessary to the VideoWrapper_vidcap code, except a recompile.

There’s one more thing to fix - it seems the video capturing is taking
up lots of CPU on my machine. I’m gonna do some debugging today to
figure out why that is.

I’m really looking forward to getting this working. wink

-Dave

-------------

Cool ... can I get those changes?

> There’s one more thing to fix - it seems the video capturing is
> taking up lots of CPU on my machine. I’m gonna do some debugging
> today to figure out why that is.

I suspect it’s the DirectShow YUV->RGB filter.  I’ve found it to be
particularly processor intensive.  Not too big a deal on modern
machines, but on some of our older (e.g., 2GHz P4) machines, it
sucked 1/2 the CPU at 30fps ...

That’s one of the reasons I like the PtGrey cameras—I can get RGB
out with much less CPU involvement.

> I’m really looking forward to getting this working. wink

Me too!

-----------------

Sure - it’s really kind of a hack though.

CVVidCaptureDSWin32.cpp

line 854 (connect function)

if (CVFAILED(this->AddMode(newMode)))
{
// Delete media type on failure, otherwise
// we’ll delete it on removal from list
LocalDeleteMediaType(mt);
} else {

if(newMode.EstFrameRate != 30)
{
newMode.EstFrameRate = 30;
this->AddMode(newMode);
}

newMode.EstFrameRate = 60;
this->AddMode(newMode);
newMode.EstFrameRate = 90;
this->AddMode(newMode);
}

-----------------

and:

CVRES CVVidCaptureDSWin32::SetMode( VIDCAP_MODE& newMode )
{
AM_MEDIA_TYPE* mt = (AM_MEDIA_TYPE*)newMode.InternalRef;

VIDEOINFOHEADER *pvi = (VIDEOINFOHEADER *)mt->pbFormat;
pvi->AvgTimePerFrame = (int)(10000000 / newMode.EstFrameRate);

// This forces the frame rate
.......

>
> I suspect it’s the DirectShow YUV->RGB filter.  I’ve found it to be
> particularly processor intensive.  Not too big a deal on modern
> machines, but on some of our older (e.g., 2GHz P4) machines, it
> sucked 1/2 the CPU at 30fps ...

I’m not sure if it’s directX or not. I’m using the I420 video mode and I
tried YUY2 and YUV, all are about the same. When I run the CodeVis
VidCapture test app it doesn’t take quite so much CPU, but videowrapper
takes up lots of CPU.doing the same thing. Any ideas? I’ve been looking
though the video wrapper code but I don’t see anything that looks like it
would cause a slowdown.

-Dave

------------------

Weird.

Is there any polling issues that could result in spinning behavior?

Are you doing any waiting in your code between calls to the video
wrapper?  Or, are you polling it repeatedly?  I wonder what’s up. 
Can you put any timing calls (getting time before/after operations)
into the code to see what, if anything is taking time?

---------------

> >> Cool ... can I get those changes?
> >
> > Sure - it’s really kind of a hack though.
> [...]
>
> Ah I see.

A cleaner solution might be to add FPS as a custom parameter.

I’ll do some more experimenting today. It takes up CPU as soon as I call
‘startvideo’ - it doesn’t matter whether I call getFrame or not.

-Dave

------------------

It’s supposed to make all the possible FPS options available.  For
video capture cards, which express a range of resolutions, we
actually made a set available.  PErhaps that needs to be done here.

------------------

I installed AMD CodeAnaylst today and did a profile against the videowrapper
example app. Here’s the top offenders:

ntoskrnl.exe (39% total cpu) - KiReleaseSpinLock - 24.4%,
ObReferenceObjectByHandle (5%).. etc..
camdrv41.sys (11%) - the camera driver, ths is normal I’m guessing.
ntdll.dll (9%) - NtReleaseMutant (4%), NtWaitForSingleObject (4%),
hal.dll (9%) - KfLowerIrql (5%), KeRaiseIrqToDpcLevel (2%)

VideoWrapper (6%) - SetFromWin32Bmp (4%), getFrame (1%), ....

So it looks like the cpu spike is some kind of hold-and-wait condition.

-Dave

Profile