Well boys and girls, I’ve been working hard last few weeks to make this great camera work under Windows.

As you may seen it before, here are the specs:

- 4 channel audio input:16 bits/channel, 48kHz, SNR 90db
- 56º or 75º Field of View zoom lens
- 2.1 F-stop, <1% distortion, fixed focus (25cm to 8 at 75º FOV)
- 640 x 480 at 60 frames/second
- 320 x 240 at 120 frames/second
- USB.0 high-speed data transfer
- Uncompressed video or optional JPEG compression

This makes the PS3Eye ideal for multitouch applications. The best part is the price $39.99! I found mine here.

Now, the main problem with this camera is that there are no drivers for Windows. The camera’s chipset info is virtually non-existent on the web.
After examining the camera internals (pictures to follow) I found that the video controller is OV534 and that the sensor is VGA type, most likely OV7720. Both of these are made by OmniVision.

I started thinking to my self: “This camera is awesome and it will be such a great and inexpensive replacement for Firefly MV and the like. If we could just get it to work under Windows...”

Initially, I started poking around with the USB trying to send some commands to the PS3Eye and see what happens…

After many long nights I’m bringing you the result:
- Full VGA (640x480) 30fps video capture test app that features uncompressed high quality raw video (60fps to follow)
- Low CPU overhead (since there is no decompression involved on the PC)
- Very low latency (1 frame time period)

The camera currently streams video in YUYV format, therefore each frame is 640*480*2 bytes.
At 30fps this amounts to 17.5MB/s which is pretty low in comparison to the total USB 2.0 bandwidth.

Most overhead that I currently have is the color conversion code that is implemented in straight C/C++ without any SIMD optimizations.
For real (MT) applications this code will go away, since we will be extracting raw grayscale image (every second byte of YUYV).

My driver exposes a camera as a device with direct access, thus eliminating the complexities and the overhead of DirectShow system.
I will be working on custom PS3EYE capture filter for use in TouchLib.
I am currently running Vista and all the code is developed and tested under this particular OS.

Driver Installation:

- First run the Install.bat in Driver directory. You might need to reboot your machine after this step.
- Plug in the PS3Eye camera. Let the Windows detect the hub and two USB Camera-B3.04.06.1 devices.
- One of these is the audio and the other one is the camera itself. After a few seconds Windows will automatically detect and install one of the USB Camera-B3.04.06.1 devices. This will be the audio device. The one that is left is the camera.
- Windows will prompt you for a driver. Cancel out of that and go to the Device Manager, right click on USB Camera-B3.04.06.1 device and chose Update Driver Software.
- From there select Browse my computer for driver software and select the Driver directory. The PS3Eye Camera driver will be installed.

After successful installation, your Device Manager should look similar to this:

Now run the PS3EyeTest.exe program, and the captured video as well as the FPS counter will be displayed.

Go, try it for yourself…

Enjoy!

~Alex

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*UPDATE* 09-05-2008
PS3EyeTest code update!
The test app is now running and capturing at 50fps full frame (640x480) resolution images!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*UPDATE* 09-05-2008
For IR blocking filter removal instructions take a look at this post.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[youtube]http://www.youtube.com/watch?v=Wf1dyiMeb4I[/youtube]

More details to follow soon!

You’d just place the camera , capture the background and off you go.

There are a few members on NUI who have bought lasers and line lenses. Alex , Fairlane , Justin.

So please , if one of you could test this baby out make a few videos , show how it performs , I think it could really help our community and jumpstart the building of more multitouch related technologies.

You can find a few more videos that give alot of details on the system here http://tshwi.blogspot.com/2008/05/more-of-microsofts-andy-wilson-video-of.html

These are the guys behind Citywall i think and they have been developing these for couple of years. Seems to be very interesting technique that recognises hand and fingers and keeps them tracked as one object.

Video:

[youtube]ur2qKbtapnc[/youtube]

I use my own code and here is result:

[youtube]hduVM9IvgUc[/youtube]

Thanks…

The TouchLib was modified (for fast blob processing) so that now runs synchronously to the camera capture rate. It also outputs current blob processing frame rate.

Here is the OSC app:

[googlevideo]-5059293264619248728[/googlevideo]

Here is the Fire demo:

[googlevideo]2348669027396447486[/googlevideo]

Here is the Photo app:

[googlevideo]-1747600458004604776[/googlevideo]

Here is the Wall Pong app:

[googlevideo]-692544185400839778[/googlevideo]

Here is the Puzzle app:

[googlevideo]8343784745267196229[/googlevideo]

Here is the Ripples app:

[googlevideo]686613493997414596[/googlevideo]

Here is the Smoke app:

[googlevideo]6317399209163209321[/googlevideo]

*** UPDATED ***

Dual IR laser blobs (camera view):

[googlevideo]6294265824056008069[/googlevideo]

Calibration process (here I use ROI feature of the Firefly MV camera @ 320x240 frame size):

[googlevideo]1621744246761173994[/googlevideo]

Many of you asked about my version of TouchLib with speed/CPU fix shown above. You can get it below and test it for yourself. Please note that I am using DSVideoLib as the input capture filter. The library was tested and it functions properly with this filter only. With other filters (such as VideoWrapper) the CPU usage may be very high, so please use it at your own risk.

Enjoy!

~Alex

Blog Post: http://nuigroup.com/log/comments/ehci/

Okay, there isn’t much to go on here.. but I am starting this to at least document the process of me building a rear DI LCD MT setup..

Currently I have
- 1 17” LCD screen completely taken apart… I will get photos of this soon.
- 1 modified Logitech webcam with the IR blocker removed and I’m using a floppy disc to filter out the visible light (and you can see some results there via a flame test)
[youtube]http://www.youtube.com/watch?v=FZmcf8Bry3k[/youtube]

- Enough parts to build a computer
- A coffee table which I’ll be removing the centre glass piece out and replacing it with said MT pc.

Here goes! I will update this soon!

I just wanted to put out a little preview of what I’ve been working on. I’ve been modding the existing Multi Touch Crayon Physics software that was done by the guys at multitouch-barcelona.com into a little game that requires the user to get the ball to the goal by any means necessary (just like the original Crayon Physics). Of course it’s multi touch, multi user, and I’m currently working on using objects (such as pens, cans and whatnot) to interact with the ball (for example, use a pen to create a bridge for the ball). It’s done in AS3 using Touchlib.

Due to some problems with my camera filter the application is currently running on a laptop screen while all the interaction is being done on the table. My new bandpass filter is on it’s way and should shortly take care of this problem, allowing me to use the LCD matrix I’ve built into my table. Remember, this is just a preview so there’s limited possibilities in the game as of yet, there’s only one very simple level, and you might spot some small bugs here and there.

I’ll post more material as soon as my table is fully operational, hopefully by then I’ll be able to share some more stuff as far as the software goes.

[youtube]http://www.youtube.com/watch?v=E7HWG87yNmM[/youtube]