1 of 3
1
Windows Kinect Driver/SDK - CL NUI Platform Preview
Posted: 19 November 2010 02:08 AM   [ Ignore ]
Avatar
RankRankRank
Joined  2008-05-08
Total Posts:  342
Administrator

<span style="font-size:20px;">Windows Driver/SDK Download: http://codelaboratories.com/nui</span>

We are proud to release the first preview of the new CL NUI Platform which consists of an SDK, API and a stable driver for Microsoft’s new Kinect Hardware (Xbox NUI Audio, NUI Camera, NUI Motor and Accelerometer) devices on Windows machines. You can review the KB articles here to get started using the SDK Samples and more.

To start we have a WPF/C# (.NET 3.5) Visual Studio 2010 Sample Application as well as C API (CLNUIDevice.h, DLL, LIB) and plan on extending the SDK similar to our CL Eye SDK which has Samples for C/C++/C#, Java and DirectShow.

Learn more about the platform or download the latest installer to get started…

Video/Screenshot of the Sample WPF Application - 2 Cameras 640x480@30FPS, NUI Motor/Accelerometer and LED Control:

CL_NUI_PLATFORM_SAMPLE.png

An overview of the platform:

CL_NUI_PLATFORM_OVERVIEW.png

The CL NUI C API:

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// This library is part of CL NUI SDK
// It allows the use of Microsoft Kinect cameras in your own applications
//
// For updates and file downloads go to: http://codelaboratories.com/nui
//
// Copyright 2010 (c) Code Laboratories, Inc.  All rights reserved.
//
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#pragma once
#include <windows.h>

#define IMPORT(type) extern "C" __declspec(dllimport)## type __cdecl

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// CLNUIMotor  API
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Motor instance type
typedef void *CLNUIMotor;

// Library initialization
IMPORT(CLNUIMotorCreateNUIMotor();
IMPORT(bool) DestroyNUIMotor(CLNUIMotor mot);

// Get device serial number
IMPORT(PCHARGetNUIMotorSerial(CLNUIMotor mot);

// Motor control
IMPORT(bool) SetNUIMotorPosition(CLNUIMotor motSHORT position);

// Get accelerometer data
IMPORT(bool) GetNUIMotorAccelerometer(CLNUIMotor motSHORT &xSHORT &ySHORT &z);

// LED Modes
IMPORT(bool) SetNUIMotorLED(CLNUIMotor motBYTE value);
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// CLNUICamera API
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Camera instance type
typedef void *CLNUICamera;

// Library initialization
IMPORT(CLNUICameraCreateNUICamera();
IMPORT(bool) DestroyNUICamera(CLNUICamera cam);

// Camera capture control
IMPORT(bool) StartNUICamera(CLNUICamera cam);
IMPORT(bool) StopNUICamera(CLNUICamera cam);

// Camera video frame image data retrieval
IMPORT(bool) GetNUICameraColorFrameRAW(CLNUICamera camPBYTE pDataint waitTimeout 2000);
IMPORT(bool) GetNUICameraColorFrameRGB24(CLNUICamera camPBYTE pDataint waitTimeout 2000);
IMPORT(bool) GetNUICameraColorFrameRGB32(CLNUICamera camPDWORD pDataint waitTimeout 2000);

// Camera depth frame image data retrieval
IMPORT(bool) GetNUICameraDepthFrameRAW(CLNUICamera camPUSHORT pDataint waitTimeout 2000);
IMPORT(bool) GetNUICameraDepthFrameRGB32(CLNUICamera camPDWORD pDataint waitTimeout 2000);
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Thanks and please share any videos of it in use or feedback while getting started with the NUI Platform.

AlexP

 Signature 

Computing is not about computers any more.  It is about living!

Code Laboratories
CL Studio Live - Video like never before
CL Eye Platform SDK

Profile
 
 
Posted: 17 November 2010 05:43 AM   [ Ignore ]   [ # 1 ]
Avatar
RankRank
Joined  2009-04-07
Total Posts:  146
Member

Oh thx, i am going to buy kinect, surely

 Signature 

AirControl MultiTouch Game: http://nuigroup.com/forums/viewthread/11178/
My MT table: http://nuigroup.com/forums/viewthread/11565/

Profile
 
 
Posted: 17 November 2010 05:57 AM   [ Ignore ]   [ # 2 ]
Avatar
RankRank
Joined  2009-05-06
Total Posts:  158
Member

awesome work alexp! i will have to get a Kinect now

 Signature 

web: http://www.nemenvisual.com
blog: http://www.purplesquirrels.com.au

Profile
 
 
Posted: 17 November 2010 06:51 AM   [ Ignore ]   [ # 3 ]
Avatar
RankRankRank
Joined  2009-11-09
Total Posts:  489
Sr. Member

yiiehaaa!! i just ordered kinect!! THX alex

 Signature 

My DSI Project:

DSI Table
Proj.Surface: 7D006 on top
2x PS3 Camera from Peau with 850BF & 3.6mm lense
Endlightend XXL (10mm)with 850nm LED strip

Projector: SANYO DWL100 ~ 950€

PC:
Core i5, 8GB Ram, HD5850

Profile
 
 
Posted: 17 November 2010 08:55 AM   [ Ignore ]   [ # 4 ]
Avatar
RankRank
Joined  2007-04-03
Total Posts:  241
Moderator

Nice work, seems to work nice and stable here

 Signature 

My multitouch blog: http://www.multigesture.net
Howto: Compile touchlib on windows XP/Vista
Howto: Compile touchlib on Ubuntu Linux
Downloads: Touchlib SVN builds

Profile
 
 
Posted: 17 November 2010 09:06 AM   [ Ignore ]   [ # 5 ]
Rank
Joined  2010-11-11
Total Posts:  7
New Member

Hi Alex, and thank you for your work, was waiting for something like that, I installed the sdk and demo.

If anybody is goin to run into problems like described below, maybe you have installed previoulsy another version of NUI devices, so You need to remove all the software, then, from device manager, unistall and remove driver software for all the 3 NUI devices, reinstall CL sdk, and have fun :D

// Crash case

When starting the application I got a crash, if i do not stop the application i can see the GUI, control the motors&led and read accelerometer values, but i got black textures.

it stops with this message :

/////

Description:
Stopped working

Problem signature:
Problem Event Name:  APPCRASH
Application Name:  CLNUIDeviceTest.exe
Application Version:  1.0.0.0
Application Timestamp:  4ce3ddd4
Fault Module Name:  CLNUIDevice.dll
Fault Module Version:  1.0.0.1112
Fault Module Timestamp:  4ce39440
Exception Code:  c0000005
Exception Offset:  00024c5d
OS Version:  6.1.7600.2.0.0.768.3
Locale ID:  1040

/////

So i tried to recompile the solution with visualStudio 10, look like it recompile succesfully but if I start debug it stops in MainWindow.xaml.cs (85,16) where we have this line :

CLNUIDevice.StartCamera(camera);

debug stops here with this message ”Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

/// end of crash case :D

Image Attachments
CL_crash.jpg
Profile
 
 
Posted: 17 November 2010 10:07 AM   [ Ignore ]   [ # 6 ]
Avatar
RankRankRankRank
Joined  2006-11-09
Total Posts:  1495
Administrator

Works great here Alex! smile cant wait to play with this in C#/WPF

@mitch A rather verbose error (thats a plus) have you tried restarting and or using other USB ports? Also what are your machine specs?

 Signature 

~

Profile
 
 
Posted: 17 November 2010 10:13 AM   [ Ignore ]   [ # 7 ]
Rank
Joined  2010-11-11
Total Posts:  7
New Member

It works perfectly and stable, I implemented sharedMemory to share disparity map with third parties ( just for fast prototyping).

I took a look to c interface, how can be accessed the minimum/maximum disparity distance ? has to be done via software or it can be setted on hardware ?

Profile
 
 
Posted: 17 November 2010 10:20 AM   [ Ignore ]   [ # 8 ]
Avatar
RankRankRankRank
Joined  2006-11-09
Total Posts:  1495
Administrator

@mitch I’m sure Alex could answer better but if your referring to the video like below I believe its thresholding grayscale depth image based on slider values:

So to fixed your error mitch you just removed older drivers?

 Signature 

~

Profile
 
 
Posted: 17 November 2010 10:56 AM   [ Ignore ]   [ # 9 ]
Rank
Joined  2010-11-11
Total Posts:  7
New Member

About removing drivers, you have to un-install drivers for NUI devices, but you also need to check the “un-install driver software”, so windows will not reinstall the old drivers when kinect is connected, just love if i can avoid people loosing time wih that.

The package really rocks, clean, working, stable and fast, nice work.

About the disparity measures, once we have the depth frame, is easy to change minimum and maximum of disparity to “select” different Z - range, but if that values can be setted on kinect board (i dont wanna believe they are static values),so u can perform a different Z mapping, is a different result, and can be fine tuned depending on space light and so on.

Profile
 
 
Posted: 19 November 2010 06:30 PM   [ Ignore ]   [ # 10 ]
Avatar
RankRankRank
Joined  2008-05-08
Total Posts:  342
Administrator
mitch_ - 17 November 2010 10:13 AM

It works perfectly and stable, I implemented sharedMemory to share disparity map with third parties ( just for fast prototyping).

I took a look to c interface, how can be accessed the minimum/maximum disparity distance ? has to be done via software or it can be setted on hardware ?

Mitch,

The depth map can be accessed directly in its raw form by using the GetNUICameraDepthFrameRAW API call. From the parameters you can see that you get 16bit unsigned value (depth) for every pixel. This is the raw data that we get from the Kinect Depth camera (after some processing done on-board by the PrimeSense’s SoC chip). Please note that even though we use 16bit data to store these values the actual usable data is only 11-bits (i.e. 0-2047).

AlexP

 Signature 

Computing is not about computers any more.  It is about living!

Code Laboratories
CL Studio Live - Video like never before
CL Eye Platform SDK

Profile
 
 
Posted: 22 November 2010 11:20 AM   [ Ignore ]   [ # 11 ]
Avatar
RankRankRank
Joined  2009-11-09
Total Posts:  489
Sr. Member

looks great! but unfortunately there is not much info on how they managed to control win7…

 Signature 

My DSI Project:

DSI Table
Proj.Surface: 7D006 on top
2x PS3 Camera from Peau with 850BF & 3.6mm lense
Endlightend XXL (10mm)with 850nm LED strip

Projector: SANYO DWL100 ~ 950€

PC:
Core i5, 8GB Ram, HD5850

Profile
 
 
Posted: 23 November 2010 12:51 AM   [ Ignore ]   [ # 12 ]
Rank
Joined  2008-08-12
Total Posts:  91
New Member

Hi;

Anyone know of an online resource to get a Kinect from, that provides international shipping?

Thanks!

 Signature 

http://www.dragonsoft.com.my
http://www.youtube.com/user/kennethrajsimon

Profile
 
 
Posted: 23 November 2010 08:31 AM   [ Ignore ]   [ # 13 ]
Avatar
RankRankRank
Joined  2007-09-13
Total Posts:  333
Sr. Member
the_goat - 22 November 2010 11:20 AM

looks great! but unfortunately there is not much info on how they managed to control win7…

Personally, I think it’s a mistake to simply port multitouch apps over to Kinect. Look how cumbersome it is to do simple stuff like dragging a picture or drawing on screen. It’s just for the cool factor, but not really useful.

Kinect is great for very different kinds of apps, especially those that take advantage of the fact that the hardware can see your whole body. Dance Central is the perfect example.

Anyway, great job as usual, Alex. Your work will surely spark awesome stuff. Thanks, man.

Profile
 
 
Posted: 26 November 2010 09:08 AM   [ Ignore ]   [ # 14 ]
Rank
Joined  2007-08-05
Total Posts:  16
New Member

It’s looks interesting, anyway it’s not clear to me if the SW is able to detect a specific part of the body, let me say the hand, and project it as a cursor on the screen. When the hand moves, you would see moving on the screen as it were a mouse pointer. This would be important for developing applications that get rid of traditional devices to perfom common tasks such as clicking and typing. I believe that without the exact pointer localization on the screen we would lose the full control of the application and thus we would limit the potentail of this interface.

Another issue is related to the stress or fatigue that an interface like that may generate to the user when we perform common tasks for minutes. It’s not a matter if you’re playing videogame, because the fatigue is part of the game. however when we work with mouse and keyboard we limit movements mostly to fingers, while in these cases arms are more involved. Then, is the Sw able to detect people movements if they are seated?

An interesting experiment that It’s worth trying is to place the controller on top of the room and pointing it to the desk where PC is placed. The camera should be able to detect hands moving on the desk. If we also added a projector we could create an interactive tangible surface without large setups used for bulding FTIR like tables.

Profile
 
 
Posted: 26 November 2010 11:41 AM   [ Ignore ]   [ # 15 ]
Avatar
RankRankRank
Joined  2007-09-13
Total Posts:  333
Sr. Member

@frav: like I said, Kinect is not meant to replace current input devices. It requires a more open minded approach, and should be viewed as a radically new interface for radically new kinds of applications. Only that mindset would allow for truly cool stuff that actually takes advantage of the technology.

Try to think outside the (x)box.  wink

Profile
 
 
   
1 of 3
1