1 of 2
1
Microsoft Announces "Project Natal" Motion Controller
Posted: 01 June 2009 04:01 PM   [ Ignore ]
Avatar
RankRankRankRank
Joined  2006-11-09
Total Posts:  1493
Administrator

?ACT=28&fid=2&aid=3484_OfT0ZjihhYxrCI7EyBM8 width=500

This thing is really interesting! Its all 3D vision based really interesting stuff. Seems like there is some decent intelligence and or marketing fluff from Microsoft wink… either way great vision hope it works and evolves… Ohh yea and looking forward to hacking it!

Read more: http://www.engadget.com/2009/06/01/microsoft-announces-motion-controller-for-xbox-360/

Compatible with any Xbox 360 system, the “Project Natal” sensor is the world’s first to combine an RGB camera, depth sensor, multi-array microphone and custom processor running proprietary software all in one device. Unlike 2-D cameras and controllers, “Project Natal” tracks your full body movement in 3-D, while responding to commands, directions and even a shift of emotion in your voice.

In addition, unlike other devices, the “Project Natal” sensor is not light-dependent. It can recognize you just by looking at your face, and it doesn’t just react to key words but understands what you’re saying. Call a play in a football game, and players will actually respond.

Image Attachments
ms12-003.jpg
 Signature 

~

Profile
 
 
Posted: 01 June 2009 04:03 PM   [ Ignore ]   [ # 1 ]
Avatar
Rank
Joined  2009-03-31
Total Posts:  71
New Member

cool!
thanks for the info!

 Signature 

Team Zugard
Jaipur, INDIA
WE HACK INTO HE@RTZ!!!
smile

Visit ZUGARD’s Blog!!!

Profile
 
 
Posted: 01 June 2009 04:15 PM   [ Ignore ]   [ # 2 ]
Avatar
Rank
Joined  2009-05-24
Total Posts:  32
New Member

Yeah, thanks for the post! Now how do I catch up because I wanna hack it too!!! The possibilities are a mathematical equation I can’t pronounce. But unlimited is a good start. cool smile
I found a link regarding the tech behind the controller > TOF (Time Of Flight)

 Signature 

due to the graphic Nature of this program, User Interaction is advised.

And remember gentlemen, day spent not making money, is a day wasted spending money.

Profile
 
 
Posted: 01 June 2009 06:37 PM   [ Ignore ]   [ # 3 ]
Avatar
Rank
Joined  2008-05-05
Total Posts:  56
New Member

Lies!! tongue laugh
“Product vision: actual features and functionality may vary”

Seriously, if they manage to make half of what is shown work decently this would be a major technology breakthrough.
I kinda hate, though, how they try to attract kids by showing cool stuff that is clearly impossible to do.

Thanks for the heads up, Christian.
Any info on what’s inside the device?

Profile
 
 
Posted: 01 June 2009 07:08 PM   [ Ignore ]   [ # 4 ]
Rank
Joined  2008-11-13
Total Posts:  22
New Member

I have tried a motion controller for a driving game before. It had gesture support and stuff like Natal. The tracking was no where near as smooth as shown in the video but this was done by a smaller company.

Profile
 
 
Posted: 01 June 2009 08:43 PM   [ Ignore ]   [ # 5 ]
RankRankRank
Joined  2008-04-30
Total Posts:  321
Sr. Member

This tech all works with the 3D vision cameras they are using that can resolve a depth matte based on what it sees. I think this system uses two of the cameras to deal with occlusion. The scene with the dress is total bullshit… unless every dress designer starts putting 3d models of their dresses into a Microsoft database. And the scene with the skateboard… if it takes a picture of the board and uses that then there is slight issue of the fingers in the way, this could easily be fixed with some good algorithms and two pictures taken with different finger placement but this video makes it look to damn simple. The TV in the video is clearly replaced and faked.

 Signature 

I’m bad about monitoring posts so if you have a question regarding something I’ve said, then just send me a PM.

Profile
 
 
Posted: 01 June 2009 10:51 PM   [ Ignore ]   [ # 6 ]
Avatar
Rank
Joined  2008-05-07
Total Posts:  74
New Member

It was just a matter of time until they commercialized it.

Here’s an old test demo that I just uploaded doing similar stuff i did a while ago. Used a Camera and distance sensors.

 Signature 

chris szadkowski
interactive design + technology + creative direction

[Motion Gesture Navigation with Depth Sensors (Complete)]
[24” FTIR Projection-Based Multi-Touch Installation (Final Stages)]

Follow me on Twitter
My Vimeo Videos

Profile
 
 
Posted: 01 June 2009 11:13 PM   [ Ignore ]   [ # 7 ]
Avatar
Rank
Joined  2009-05-24
Total Posts:  32
New Member

@Cris, that’s rather cool boss! Did you take it any further?

 Signature 

due to the graphic Nature of this program, User Interaction is advised.

And remember gentlemen, day spent not making money, is a day wasted spending money.

Profile
 
 
Posted: 02 June 2009 01:23 AM   [ Ignore ]   [ # 8 ]
Avatar
Rank
Joined  2008-05-07
Total Posts:  74
New Member
Nui-NooB - 01 June 2009 11:13 PM

@Cris, that’s rather cool boss! Did you take it any further?

I did. I create a piece for the AdAge Dinner in NY last year to demo some of my companies work. It was very similar to the new xBox demo with Skateboard decks and the Netfilx Video player at the end. But i really never got a chance to do anything really exciting with it.

I’m excited to hack this and see what awesome things we can do with it.

 Signature 

chris szadkowski
interactive design + technology + creative direction

[Motion Gesture Navigation with Depth Sensors (Complete)]
[24” FTIR Projection-Based Multi-Touch Installation (Final Stages)]

Follow me on Twitter
My Vimeo Videos

Profile
 
 
Posted: 02 June 2009 08:47 AM   [ Ignore ]   [ # 9 ]
Rank
Joined  2009-06-02
Total Posts:  6
New Member

Cool ! Time Of Flight .  It’s time to say goobbye to FTIR , DI , RI display surface .

I wonder if we use normal camera ? Or We must use special camera with sensor hardware embbeded inside.

Profile
 
 
Posted: 02 June 2009 10:20 AM   [ Ignore ]   [ # 10 ]
Avatar
Rank
Joined  2008-05-07
Total Posts:  74
New Member

I think Microsoft worked with GestureTek and Canesta to build it. More info check out their press release about it from a couple of years ago: http://www.gesturetek.com/press/press_may10_06.php

They used vision system from Canesta: http://www.canesta.com/.

http://news.cnet.com/8301-17938_105-10253925-1.html

 Signature 

chris szadkowski
interactive design + technology + creative direction

[Motion Gesture Navigation with Depth Sensors (Complete)]
[24” FTIR Projection-Based Multi-Touch Installation (Final Stages)]

Follow me on Twitter
My Vimeo Videos

Profile
 
 
Posted: 03 June 2009 02:36 AM   [ Ignore ]   [ # 11 ]
Rank
Joined  2009-05-11
Total Posts:  17
New Member

here’s some more info on the tech side.
http://procrastineering.blogspot.com/2009/06/project-natal.html

since its not shipping, who knows how good it really works or when it will actually arrive. on the other hand, they have chosen to play their cards to generate a lot of press and pique our curiosity. if it can be done, its of great interest many in this group. lets find out how.

best,

-robert
F ^ S H I O N B U D D H ^

Profile
 
 
Posted: 03 June 2009 03:44 AM   [ Ignore ]   [ # 12 ]
Rank
Joined  2009-06-02
Total Posts:  6
New Member

Ha ha ! Johnny lee (who did low cost touch table with wiimote ) is a team member of Project Nathal . Cool ! 

Anyway , I wonder that if Microsoft want to take cover all licenses on Time of Flight tech ?  I know that Microsoft bought 3DV (one small company has Time of flight tech) http://en.wikipedia.org/wiki/ZCam .  Big M$ also bought licensees from GestureTek on Time of flight tech , in this webpage http://www.gesturetek.com/3ddepth/introduction.php , look at demo , it is the same Project nathal , exactly , and scrool to the bottom of the webpage you can see EYETOY and XBOX360 logo . So , what can I say ? I think sony bought licensees for ps2 eyetoy , but that gesture tech version is the old one, just 2D gesture . In the case of project Nathal , It can sense 3d object , so this gesture version is the new version . And I think big M$ bought exclusive licensees for long term (maybe 10 years ..) , adn they also bought 3DV company . So, Micrsoft will be the winner in next generation console war .  Oh man ! who know . Sony and Nintendo can’t purchase Time Of Flight tech due to exclusive licensees .

Profile
 
 
Posted: 03 June 2009 03:48 AM   [ Ignore ]   [ # 13 ]
Rank
Joined  2009-06-02
Total Posts:  6
New Member

This is how 3d sensor chip capture the light and produce the pixel to render 3d model :

howitworks.jpg

Most people understand that light takes a finite time to travel between two points—that photons of light from two different stars, for example, may have started their journeys years, or even millennia apart. Since light travels essentially at a constant speed, if you know the time, you can calculate the distance.

The light illuminating each individual pixel in an image sensor comes from a different feature in the scene being viewed. Canesta recognized that if you could determine the amount of time that light takes to reach each pixel, you then could calculate with certainty the exact distance to that feature. In other words, you could develop a three-dimensional “relief” map of the surfaces in the scene. In three dimensions, objects previously indistinguishable from the background, for example, metaphorically “pop” out. For a broad class of applications, this proves extremely helpful in reducing the mathematical and physical complexity that has plagued computer vision applications from the start.

In a recently-granted U.S. patent, Canesta describes several of its inventions for “timing” the travel time of light to a unique, new class of low-cost sensor chips.

Fundamentally, the chips work in a manner similar to radar, where the distance to remote objects is calculated by measuring the time it takes an electronic burst of radio waves to make the round trip from a transmitting antenna to a reflective object (like a metal airplane) and back. In the case of these chips, however, a burst of unobtrusive light is transmitted instead.

The chips, which are not fooled by ambient light, either then time the duration it takes the pulse to reflect back to each pixel, using high speed, on-chip timers in one method, or simply count the number of returning photons, an indirect measure of the distance, in another.

In either case, the result is an array of “distances” that provides a mathematically accurate, dynamic “relief” map of the surfaces being imaged. The image and distance information is then handed off to an on-chip processor running Canesta’s proprietary imaging software that further refines the 3-D representation before sending it off chip to the OEM application.

Profile
 
 
Posted: 03 June 2009 10:28 AM   [ Ignore ]   [ # 14 ]
Rank
Joined  2009-05-01
Total Posts:  3
New Member
gsoto - 01 June 2009 06:37 PM

Lies!! tongue laugh
“Product vision: actual features and functionality may vary”

Seriously, if they manage to make half of what is shown work decently this would be a major technology breakthrough.
I kinda hate, though, how they try to attract kids by showing cool stuff that is clearly impossible to do.

Thanks for the heads up, Christian.
Any info on what’s inside the device?

gsoto,

Frankly, almost all of what’s shown in that video already exists.
The karate and monster parts are totally doable and 3DVS has a boxing demo that’s pretty similar, http://www.3dvsystems.com/gallery/gallery.html
The driving part has already been demoed at E3 with Burnout
The football part is just the same as the karate/monster part but it’s tracking two players instead of one (and the tech right now is supposedly able to track 4 players so 2 should be no strech)
The skate scanning part has been demoed at E3 with the Milo project (you can draw to Milo and have it scanned by the webcam - of course that the blue highlight in the promotional video is total bs)
The dress part is there to highlight facial recognition and Xbox Live automatically logging in after it sees your face (and facial recognition will be greatly improved due to the the combination of a 3D + 2D camera as it will not only rely on a “photo” but will also analyze facial features, eg. nose size)
The 1 vs 100 part will highly depend of the ability of the system to properly track multiple players
The movie part, browse through menus and use gestures to control, is already doable and the tech has been demoed at E3

Afaik, what’s inside is merely a IR cam, IR light source, HD webcam and multiple microphones. Plus the hardware to make it all work like magic smile

tetsu81vn - 03 June 2009 03:44 AM

Anyway , I wonder that if Microsoft want to take cover all licenses on Time of Flight tech ?  I know that Microsoft bought 3DV (one small company has Time of flight tech) http://en.wikipedia.org/wiki/ZCam .  Big M$ also bought licensees from GestureTek on Time of flight tech , in this webpage http://www.gesturetek.com/3ddepth/introduction.php , look at demo , it is the same Project nathal , exactly , and scrool to the bottom of the webpage you can see EYETOY and XBOX360 logo . So , what can I say ? I think sony bought licensees for ps2 eyetoy , but that gesture tech version is the old one, just 2D gesture . In the case of project Nathal , It can sense 3d object , so this gesture version is the new version . And I think big M$ bought exclusive licensees for long term (maybe 10 years ..) , adn they also bought 3DV company . So, Micrsoft will be the winner in next generation console war .  Oh man ! who know . Sony and Nintendo can’t purchase Time Of Flight tech due to exclusive licensees .

tetsu, your assessment is pure fud. We know for sure that GestureTek licensed their tech to more than MS (Sony itself is included with their EyeToy), so no need for irrational fear, ok wink

Profile
 
 
Posted: 03 June 2009 05:00 PM   [ Ignore ]   [ # 15 ]
Rank
Joined  2009-06-01
Total Posts:  3
New Member

Very exicting possiblities for hacking though. Full body motion capture for a (lower) price without any kind of sensors? This could be very good for animators.

Profile
 
 
   
1 of 2
1