Microsoft introduces controller free gaming, Project Natal

Microsoft today introduced Project Natal, a method for interacting with games, controller free.

The software giant unveiled the project at a packed out press event for this years E3 gaming expo in Los Angeles, California. The ground breaking technology was introduced by both Stephen Spielberg and gaming legend Peter Molyneux. Both believe the technology has the power to turn games into a true mass entertainment medium. Spielberg said "This is a pivotal moment that will carry with it a wave of change, the ripples of which will reach far beyond video games".

The technology behind Natal is fairly simple, a TV-mounted camera/microphone bar to sense motion, sound and movement. What's new and ground breaking is the tracking of 3D movement as well as 2D movement. The system is also able to scan users faces and sign them into Xbox live, as well as react to vocal tone and pitch, and process this data in real-time. The key to the system is accuracy and during one of the demos the presenter managed to move an avatar in a rather bizarre and un-human like fashion:

Regardless of the flaw in the demo, the technology looks promising and will allow users to interact with the Xbox menu systems, avatars and games specifically designed for the upcoming unit. Microsoft confirmed Natal, which has no official name, release date or price, will also work with existing Xbox 360 consoles as well as all future models.

Report a problem with article
Previous Story

Live blogging of Microsoft's E3 Press Conference [Finished]

Next Story

Metal Gear Solid Rising for Xbox 360 revealed

123 Comments

Commenting is disabled on this article.

Playing a game such as COD would be most interesting, although it would have to come with a controller (LMG or something).

as I understood, this is suppose to release in 2010? "Natal" means "Christmas" in portuguese. Has this any relation to the release date of this?

This won't replace controllers since you couldn't walk around in a virtual world, and the camera would probably not be able to detect the motion of your index finger as you squeeze that trigger.

Dude, If they remap the software to Windows for Natal, then the two of these together would be awesome.

I was thinking about a tilt board like the new tony hawk skateboard to control movement in a FPS.

Tilt forward to walk/run forward,
Tilt to the side to strafe,
Twist to turn,

and let natal extrapolate the aiming from you body position, and change guns either based on verbal commands or by the posture which you are mimicking. (IE, you don't hold a bazooka/BFG the same way that you would hold a handgun)

i cant get excited about it. its basically a beefier version of webcam games that have been around forever.

WOW i wasnt expecting anything as advanced as this, this realy does make the wii look like a childs toy.
i wounder if sony have been dveloping some thing like this too.

i may just be tempted to buy an xbox360.

fantastic work microsoft.

It scanned in the image that quickly? And recognized it...

This makes the Nintendo Wii look like a baby's toy... Let's hope they get it to actually work like they have demonstrated in the videos without lots of miscue's.

Just insane!

how is it that a computer understands what the girl is saying and can formulate sentences that make sense? Are there specific scenarios? Or can u ask mylo any question and he has a response?

If I had to guess, I would say it's following certain patterns that make sense to us without having any real content. Most conversation is empty speech, after all. For example, when she said "thousands of people" with a worried tone of voice, all Mylo does is repeat it with a worried tone of voice and expression. It doesn't mean he relates, but it does mean he was programmed to know when to do the 'worried echo' that people do in real conversation a LOT.

Also, when she's playing with the fish in the water, the things Mylo says aren't in response to anything she says, and could just be basic NPC-speak triggered by the processing of her image as a reflection in the surface. Likewise, when she calls him 'cheeky' for his joke, he doesn't directly respond to that.

There's some very clever code at work, and clearly a lot of research was put into conversation habits and patterns.

I love this new technology..I love what microsoft is doing here and I cant wait to try out the technology. I wish I could have gone to the E3 conference but tt seems to me though that the prospect of this type of gaming could turn out to be scary as this report indicates: http://bit.ly/109RxI

This really doesn't interest me very much. The concept is somewhat exciting, but I think the reality will be a bit of a letdown. As other users have mentioned the accuracy will play a big part of the peripheral's success. I really enjoy playing Eyetoy Groove, but it does miss some motions and the accuracy is highly dependent on the amount of light in the room, how well it is dispersed, and the contrast between your skin and your furniture. Granted the quality of the camera in the Natal device is probably a few steps above that of the EyeToy, and since it claims to be able to detect motion in three dimensions, I am guessing there's a special light and a second camera inside the unit to help detect depth.
Will this really be very fun? The driving game really caught my attention. I really don't think it would be very fun to have to hold your hands out in front of you like the girl does for any length of time. The Wii really led the way with its new controller paradigm, but it didn't take me very long to grow tired of the novelty and forget about the system after finishing Twilight Princess and Mario Galaxy. A different way of controlling a game can only do so much. I can't see this device really taking us much further than the EyeToy and the Wii already have.
There is one thing to look forward to, however. Remember all those morons who threw their Wii controllers into their televisions? Imagine all of the kicked plasmas and kicked faces from little ninja turtles playing those fighting games. Can't wait to hear about that!

This controller free tech is not bad, no more spending on extra controllers!
Unless they need some online activation($$$) for extra players.

But I think that a physical controller will still be more accurate.

Looks cool. But, practically, it is going to suck. There are only so many humanly possible things you mimic so that they're performed in-game; but what happens when you want to play a game like Devil May Cry, Ninja Gaiden, or any non-realistic fighting game (Tekken, Soul Calibur, Virtua Fighter)? It's physically impossible to pull off some of the insame moves in those games, and hence is impossible to mimic effectively on this device.

So, it then boils down to two possiblities. 1, the "controls" for those games degrade in to a person doing generic gestures to do the over-the-top moves, which in my mind, defeats the whole purpose of the idea. or 2, games in which fantastical action and gameplay are marginalized and "realistic" games become the norm. I don't know about you, but both of those possiblities are terrifying.

This technology would be sweet for controlling a computer's OS (a.k.a Minority Report), but games? No way.

spacer said,
Looks cool. But, practically, it is going to suck. There are only so many humanly possible things you mimic so that they're performed in-game; but what happens when you want to play a game like Devil May Cry, Ninja Gaiden, or any non-realistic fighting game (Tekken, Soul Calibur, Virtua Fighter)? It's physically impossible to pull off some of the insame moves in those games, and hence is impossible to mimic effectively on this device.

You'll pick up the xbox360 gamepad and play the game. Obviously this isn't practical for EVERY game as a main controller, but the possibilities are endless to enhance the game (put you into it).

howb21 said,
Ermm...you all do realise that means you can have simulated sex in video games right?


I'm just waiting for the USB powered pocket.... errrr... you know where I'm going with this...

yeah this is a good idea for some things and definitely the way forward except for things that require more physical feedback, for instance the racing game seems far to awkward and FPS would benefit from have solid props to use. Same to be said for the swinging of a sword / stick.

i can't seem to click on 'reply to this comment' argggh. anyway, i want this on the pc! i've always wanted to control my media center or pc ever since i saw minority report.

i wonder how they're going to handle rumble/vibration feedback? visually?

The Wii was a revolutionary system because nintendo approached gaming from a sideways perspective. The centerpiece of the Wii is this new, highly tactile interface. It's intuitive nature was a draw to a whole new kind of target market, opening up huge demographics that they, essentially, still have a monopoly on because theirs is the only product that can cater to it.
This is Microsoft's stab at that market share, their next step away from the world of buttons. I guess this could turn out well for the consumer, and on paper, it usually does for microsoft, but I've gotta say I'm a pessimist. I never really trust them to produce something solid. I see this as a formulaic, if-then profit formula from microsoft, versus the out of the box thinking at nintendo.
And that video really doesn't do them any favors, either.

This here is the next evolution in gaming. Nintendo got the ball rolling, but this is great.
This is the kind of thing that gaming needs, something that everyone can do, and its fun. It really gets you off the couch and its easier to figure out than a controller.

If they can do it, I'm shelling out for a 360 and Natal. But until then, call me a doubter. That is extremely cool though but I struggle to see it ever beating the controller completely, as you cannot get many complex commands like that. But I mean, features like voice recognition for stuff really is going to great.

And why must they not reveal the answer to "How many push ups Chuck Norris can do?", I mean, it is obviously infinite.

If Microsoft can pull it off, if it does really work, it's going to be Bye Bye Wii....

But you can have the best technology, you still need good games. Many good games needs to come out fast using "Natal".

Imagine if Windows 8 used the next gen 'Natal' technology. You wont even have to put finger prints all over your lovely display!

Imagine if Windows 8 used the next gen 'Natal' technology. You wont even have to put finger prints all over your lovely display!

Looks promising, I'll be sure to keep informed about this. I won't buy it because I don't have nor want a 360 nor have the money for it, but it gives a good idea of what next-gen consoles will be able to do. Sone's EyeToy and the Nintendo Wii were nice tries, but not as good as we thought maybe, now let's see what Natal has in his guts...

"Product vision: actual features and functionality may vary"

Sorry but IT IS NOT POSSIBLE, not at least how is showed in the video, for most, it will become a powered eyetoy and thats it!.

:-/

Magallanes said,
"Product vision: actual features and functionality may vary"

Sorry but IT IS NOT POSSIBLE, not at least how is showed in the video, for most, it will become a powered eyetoy and thats it!.

:-/

O rly? You working with the MS team on this or work in the same field and have insider info that would show this is immposible? Untill it comes out or working demos for people to use hit conventions its way to early to say that.

I can guarantee you that this has been possible for a very long time.

It's just now becoming practical for a consumer price point.

"how many push-ups can Chuck Norris do" , haha.

the face recognition login, and voice recognition is awesome, and if it reacts as smoothly as in the presentation even better.

its looking amazing for 'simple' gaming, id like to see how the do the controls for games like halo or COD.

The Edmonton science centre had something like this years ago. Of course, not anywhere near as advanced and that was quite fun. Will have to see how it does though.

That's weird, my mind went to Sony's eye toy rather than anything Nintendo ever did.

Randomly accusing MS of stealing from someone they weren't even paying attention to? Ew, Nintendo users have become Apple users...

/Microsoft was paying attention to camera gesture recognition before the Wii hit the market. Hell, Logitech webcams came with cheesy webcam games back in the 90s.

emmi said,
Microsoft finally acknowledged that Nintendo Wii is a hit so they want a piece of the pie. For me this is way too childish.

Playing games with a controller is getting old. I want full interaction with my games and this looks to bring it how is that childish?

emmi said,
Microsoft finally acknowledged that Nintendo Wii is a hit so they want a piece of the pie. For me this is way too childish.
Finally? They acknowledged it years ago? Infact I think they went as far to say they see them as their biggest threat, not Sony, at one point

This might become something like HAL 9000 controlling Skynet (God, I'm old...! ) but really looks amazing. I just can't imagine an RPG with this, but I'm sure they'll come up with something.

RPG designers are generally lacking in imagination when it comes to gestures. Rather than keeping it simple, they have a habit of overcomplicating things (DS stylus usage, for example) to the point where it becomes more about dexterity than wits. They'd expect us to memorize two or three dozen different combinations of hand gestures (think Naruto) to use any spell or skill.

While there are no doubt ways to make RPGs work with this, I get the feeling developers will not figure them out, ruin the genre, and used PS2 sales will stay steady for decades.

Gladiatorus said,
This might become something like HAL 9000 controlling Skynet (God, I'm old...! ) but really looks amazing. I just can't imagine an RPG with this, but I'm sure they'll come up with something.

I think a fps style rpg like elder scrolls would be freaking amazing with this. Pulling back the string for the bow, maybe making a certian gesture for a spell could be epic.

Yeah, I was just thinking that. Imagine a new way to handle spells. To have a verbal incantation that has to be spoken along with semantic gestures. That way, you don't have to "lock" spells to specific levels, you can simply have them be discovered during education or from other wizards along your travels.

Rock on, I ran across the touchless api a couple of months ago and thought how cool it would be to use it to make stuff like that. What they've done for the x-box is light years ahead of it though.

This is straight out of a science fiction, it was really hard to believe !
One thing though .... Microsoft will have to be very careful, the same thing that happened to Nintendo when it brought out the Wii will happen with this ! The Americans will be in "SUE" mode when they hurt themselves and or damage objects such as their TV!!
Nintendo had several cases where stupid Americans threw the "Nunjuck" at the TV screen (Hence the wrist strap... which still didn't work on some idiots).

Caveman-ugh said,
This is straight out of a science fiction, it was really hard to believe !
One thing though .... Microsoft will have to be very careful, the same thing that happened to Nintendo when it brought out the Wii will happen with this ! The Americans will be in "SUE" mode when they hurt themselves and or damage objects such as their TV!!
Nintendo had several cases where stupid Americans threw the "Nunjuck" at the TV screen (Hence the wrist strap... which still didn't work on some idiots).

That's one of the advantages of "controllerless". What are you going to throw at the screen exactly? :P

This looks absolutely freaking amazing. If MS can pull this off it will revolutionize interacting with a console/pc whatever. Good One MS!!!

Looks great - how it'll work in real living rooms is another matter with clutter around etc - be interesting to see. Thing is - it is another device announced but probably not out til next year. No games developed for it as dev kits only going out this week. Hopefully we see some decent stuff when it comes out next year unlike what happened with the Wii for the first 6-12 months.

I don't think that xbox have hoursepower to do it. something in betwewen it's needed.
the question is how much will it cost ? 200 bucks ? 300 ? maybe next xbox can have it built-in.
is it real or it's a demo, look too much amazing for me but no release date, no price... too good to be real ?

I think thats the whole point behind this technology. Back when I was a kid we played outside from dawn til dusk. As time passed and more kids got snatched off the streets by perverts and early video games became available on the very first home computers and standalone gaming units like Atari; parents allowed and eventually encouraged their kids stay indoors to be safe.

The result is a culture of freightened children that get little exercise and face the reality of becoming lazy and overweight.

This technology has the potential to change that by encouraging the entire family to get mobile without even leaving the home. Granted, it won't replace true outdoors activities real soon but in time it may.

Anyone remember virtual reality headsets, or more recently a thing called a Wii it will all go the same way another gimick

Boramas said,
Anyone remember virtual reality headsets, or more recently a thing called a Wii it will all go the same way another gimick

You mean the same Wii that is making Nintendo money by the bucketloads? That Wii?

You may go now.

Boramas said,
Anyone remember virtual reality headsets, or more recently a thing called a Wii it will all go the same way another gimick

I would hardly call the wii a gimmick.

Boramas said,
Anyone remember virtual reality headsets, or more recently a thing called a Wii it will all go the same way another gimick

I'm not so sure. This doesn't require the user to wear anything or hold a weird controller, or do anything other than be in front of a properly equipped display.

lylesback2 said,
Oh... My... God!

I am almost speechless. That is the best thing EVER!


I have to agree. I can't see anything topping this. If it works as good as it does in the video, I might actually have to buy an Xbox.

Yeah I just saw the other video where the avatar spags out, and it doesn't seem as perfect as this video. Hopefully when it does come out, all the kinks are ironed out.

Manish said,
If it works as good as it does in the video, I might actually have to buy an Xbox.

It won't have the kind of precision shown in the video.

Hmm. Well I didn't think it would be perfect. They've got a year or so to make improvements though. The technology can only get better so eventually it will get to that level of precision.

Looks amazing! The next evolution of gaming is doing away with controllers and immersing yourself in the game. Nintendo made a good first step with the Wii but this takes it and totally blows it away.

Anyone got any close-ups on the device or info about camera lens/resolution and the microphone in it?

I think the purpose of gaming is what is on the screen _in the game_ and not what the person is doing while playing the game.

I didn't know games didn't exist outside of computers. I guess I must have been imagining tag. If you think in screen is the most important aspect, I suggest you stick with turn based strategy as anything else requires reflexes, as minimal as they are in a games controller.

I always hated the idea behind these types of devises, too much effort for something a controller can do (like the girl driving the car) but if Microsoft can pull this off in the way it says this will be truly awesome

Driving the car did not look like it was a lot of effort but I would like to have a steering wheel controller with real force feed back and not the crappy rumble function.

well it's effort for something a controller can do but at the same time it allows you to do things a controller can't (and likewise you can do things with tactile buttons that you likely won't be able to replicate using this)

StaticL said,
I always hated the idea behind these types of devises, too much effort for something a controller can do (like the girl driving the car) but if Microsoft can pull this off in the way it says this will be truly awesome


Just like driving and maintaining a car is so much harder than walking. Why not just walk?

Holy crap that looks fun! Can't wait to get my hands on something like that. Hopefully if it's a single player game the camera doesn't pick up people in the background trying to screw with you.

Of course it will. The question is whether the software can filter it...assuming you can't just go ahead and kick the butt of your "friends" in the background. 8D

I'm an instant fan. I agree with WolfDV that we'll see about it's accuracy but there are just a bunch of casual uses for this. No remote needed to pick a movie.

I'd love to see them turn this into a PC accessory like they did with XBox Controller.

+1 Games, fine. Replacement for upcoming touch screen fiasco...oh yes!

No need to replace your monitors and big screen TV with this tech. 8D

excalpius nailed it. Touchscreen has been monstrously overrated for large-scale use. This camera recognition stuff will win the market in a landslide due to affordability.

True true. Minority report tech works best THIS way...you can even control windows, etc. in a normal seated position at your desk. 8)

yea the racing game was pretty hilarious haha. I think it was just to show that they can use Natal with anything.. or at least i hope so.

Amazing technology there but will it really take off? What games or applications can really use that other than the ones shown? We'll have to wait and see I guess. It looks awesome though.

Well it'd allow you to do anything the Wii can presumably. It's not like they couldn't give you some kind of joypad to allow additional commands too.

I really don't expect developers to support this in the masses, likely a select few games that support it. Accuracy is going to be no where that good, plus look at their setup, few people have it like that. I assuming lighting is going to be important too.

And driving, ugh.. epic fail. What about shifting? Gas? Am I supposed to look behind me for it to show me what's behind me? lol It's a cool idea sure, but call me old fashioned I like holding something, even if its a wiimote.

looks interesting; like what sonys eyetoy should have been.

we'll see when released how accurate it actually is.

True true. It would take a lot of training on our parts and the software to recognize the words and how to speak properly. There are a lot of people out there that slur or have other issues that make it hard for software to understand.

Wolfmother. I like it. That aside, this looks pretty epic, especially the skateboarding game. The people in the car one looked hilarious though, heh.

It all depends on the developers and what sorta games they make or how they use it in their games. But the technology behind it is great. Full 3D motion capture, face recognition, voice recognition. Just wow.