Microsoft developing a truly immersive gaming experience

With the introduction of the Wii in 2006, Nintendo introduced a new way to immerse users in certain games. In 2010 Sony followed suit with the Move, but it was the Kinect from Microsoft that took the next step in giving the user a truly immersive experience, removing the need for physical controllers and making you control the in game characters and objects using whatever body part you could move!

Talk of the next Xbox is growing and the next version of Kinect is being teased as well, adding more functionality to the device. Over at Patent Bolt, what could possibly be the successor (or addition) to Kinect has been detailed, following a recent Microsoft patent application.

There is only so far graphics, sound, interface and controls can go with the current (and likely next) generation of consoles. And now Microsoft are looking to push the boundaries of the immersive experience they started with Kinect, turning your room into the game environment. The patent states:

An immersive display environment is provided to a human user by projecting a peripheral image onto environmental surfaces around the user. The peripheral images serve as an extension to a primary image displayed on a primary display.

The Xbox will still work in the same way, playing the game on a TV, while an environmental display will also project the game environment, seen outside TV screen boundaries, on surfaces around the room. The images show a user playing a first person shooter, but imagine being able to see enemies, literally out of the corner of your eye, while taking down the ones in front.

With Kinect to continue capturing your movements, a depth camera capturing your gaming space and the possibility of using 3D technology (using either 3D glasses or a TV with suitable parallax barriers) to bring the TV experience that bit closer to the user, it’s the next obvious step in bringing the user closer to their games (or bringing the games closer to the user).

Microsoft has teased holographic displays and interaction through their Research department, and with this patent application it looks likely that’s where they are ultimately going for the gaming experience. Although putting a timescale on the technology getting to the home is another question entirely.


How many of us are thinking "The holodeck in my lifetime? Yes please!"

By now, I'm sure we've all seen how the Wii U uses the tablet controller to allow the user to see what's outside of the screen, but this would take things to a whole new level.

There are many more details to wade through, such as environment colour compensation and how to keep the gamer from being mistaken for a piece of furniture! Hit up the source for all the information.

Source and images: Patent Bolt Thanks to Sandaror for the tip!

Report a problem with article
Previous Story

TechSpot: Traveling with an iPad: Impressions & Accessory Survival Guide

Next Story

A new 4G network for the UK, and a new brand: EE

30 Comments

Commenting is disabled on this article.

I can imagine Apple started taking interest in this and start making hardware and claim it's innovation just like touch screen and mouse.

Maybe initially it will be an arcade-based game, like real life arcades at malls and such. That would be cool, and would give Microsoft and arcade owners a consistent money source if it pans out well. Sometimes, not every bit of tech is accessible as a personal consumer product.

Social experiment:

Find 200 gamers who haven't heard about this yet. Split them into two groups of 100.

Tell the first group that Microsoft is developing this tech. Gauge positive/negative reactions.

Tell the second group that Gabe and Valve are developing this tech. Gauge the same.

Joshie said,
Social experiment:

Find 200 gamers who haven't heard about this yet. Split them into two groups of 100.

Tell the first group that Microsoft is developing this tech. Gauge positive/negative reactions.

Tell the second group that Gabe and Valve are developing this tech. Gauge the same.

There is more anti-Microsoft sentiment than people that even know who Valve/Gauge is.

If your point was it would fail it didn't have the Microsoft name, you are kind of not thinking this through. Even with the XBox 360, the 'Microsoft' name was not something people attached, and even 'self' regulated in their conversation.

Notice games not mention Microsoft or how awesome Microsoft is, instead listen to them never mention Microsoft and still talk about how awesome their 360 is.

I think a couch and a couple of friends will do. It's been that way since the original Nintendo. Why screw that up with a cubicle?

More **** we don't need.

bjoswald said,
I think a couch and a couple of friends will do. It's been that way since the original Nintendo. Why screw that up with a cubicle?

More **** we don't need.


You must be a hoot and a hollar at parties.

People don't NEED games, either, yanno. That's never what this is about. I honestly don't understand people like you who look at advances in technology and utterly FAIL to see the potential directions it could be taken in, instead preferring to get completely wrapped up in the initial application of the technology, judging that against your own tastes, and then dismissing a potentially whole new industry on those grounds alone.

It's shortsighted and selfish, frankly.

bjoswald said,
I think a couch and a couple of friends will do. It's been that way since the original Nintendo. Why screw that up with a cubicle?

More **** we don't need.

Pong is good enough, hand eye coordination is all it is about anyway. Those new fangled Pac People games are just too colorful and confusing.

NeoPogo said,
Great. Microsoft expects all to play Xbox and Kinect games in a closet! No way.

Or a 30 foot screen with a wrap around on the walls of the room. This has NOTHING to do with playing in a closet.

Even 'outside' throw up a couple of wires and hang a couple of sheets.

Think bigger/outside the box.

It is just a concept and patent is being applied for. It doesn't mean that everything with this idea will happen or be included into the final product. It is just like a concept vehicle, the majority of concept vehicles doesn't make into production because only a handful of ideas within that vehicle will make it to production vehicles.

Granted, it would be nice to have something like this, and especially this, if you are playing first person shooter (http://tngames.com/products).

This assumes most people actually WANT to stand up and jump around and annoy any neighbors that might live in the apartment under them. Like Wii, people bought them, but they would rather sit down.

This type of Gaming isn't going to replace wanting to sit down and play a game and relax.

Mike Frett said,
This assumes most people actually WANT to stand up and jump around and annoy any neighbors that might live in the apartment under them. Like Wii, people bought them, but they would rather sit down.

This type of Gaming isn't going to replace wanting to sit down and play a game and relax.


So you're saying the holodeck would fail and the matrix would be a huge success.

Mike Frett said,
This assumes most people actually WANT to stand up and jump around and annoy any neighbors that might live in the apartment under them. Like Wii, people bought them, but they would rather sit down.

This type of Gaming isn't going to replace wanting to sit down and play a game and relax.

You assume most people live in a poorly made apartment.

In America, some construction is good, some is crap. I have lived in condos that you could have a Rhino running a jackhammer and not disturb the people below you.

I don't want to be dismissive, as it does suck when you can't 'play' depending on where you live.

Because I have also lived in a nice townhouse that normal volume on a TV could be heard two buildings down. (Well known area on a Golf Course in San Diego even.)

However, I have even lived in old places and was surprised how sell made and sound proof they were. When I lived in Antwerp or Brussels, we could do body hits to the floor and you could not hear it in the flat below you.


So, I feel for you, but there are a lot of people in homes and a lot of people that don't have to worry about 'vibration' or noise. These are the same people that buy the Bose sound system that can power a stadium and get to enjoy cranking it up.


Suggestion... Make your XBox/Wii/whatever a bit more of a portable solution. Save up for a projection screen or even a good 32" LCD that you don't care if it gets some dirt/sand on it.

Then head to your local beach, woods, wherever with your friends, and game outdoors or at another friend's house that noise is not an issue. When I was younger (heck just a couple of years ago) I flew everywhere with my XBox as carry on, and had a 7" LCD, 14" LCD, and even a 27" LCD that I could throw in a suitcase, and we gamed no matter if I was in Denver in a hotel or on a beach somewhere or a friends house, and we played multi-player Halo and didn't want to use split screens.

Youth, use it...

There was a video around some time ago where someone built a small dome with 360º projections, a mechanical floor that kept you at the center of the room while you were running around, a gun with motion tracking (I think the used an attached iPhone, which also doubled as scope) and softball guns to make you feel the impacts.

Quite infeasible at home, but cool stuff anyway

ichi said,
There was a video around some time ago where someone built a small dome with 360º projections, a mechanical floor that kept you at the center of the room while you were running around, a gun with motion tracking (I think the used an attached iPhone, which also doubled as scope) and softball guns to make you feel the impacts.

Quite infeasible at home, but cool stuff anyway

The video is out there, for a while Apple fans were posting it claiming the iPhone was the key to the technology, obviously not getting the iPhone was just a simple tricorder in the analogy, and had nothing to do with the technology.

Coordination of the floor, projection, sound, controls took a bit more computing than average users, but was not out of the realm of possibility.

Give it 10 years, and the Holodeck will at LEAST be at that level.

However there are several force field and other electromagnetic concepts converting the air around us into temporary solids and colors that are far more promising for a real HoloDeck experience, and we could see that in 20 years.

It's a great idea. It's just a shame that most people probably don't have simple room layouts like that. It might still work, though.

I doubt we will see this until the next next gen however I would dedicate an entire room to it.
It still frustrates me that Microsoft will not allow users to hold items. If I am playing a first person shooter I want to hold a controller in the shape of a Gun. I also want that controller to have a thumb stick that allows me to move forward and backwards. While the kinect can do alot of neat things true player movement is not one of them. A game like star Wars kinect was ruined because of ****ty player control.

ogar said,
I doubt we will see this until the next next gen however I would dedicate an entire room to it.
It still frustrates me that Microsoft will not allow users to hold items. If I am playing a first person shooter I want to hold a controller in the shape of a Gun. I also want that controller to have a thumb stick that allows me to move forward and backwards. While the kinect can do alot of neat things true player movement is not one of them. A game like star Wars kinect was ruined because of ****ty player control.

I think having items specific to some games to use with kinect will come next gen. Just wait for the custom CoD kinect rifle and watch it sell like hot cakes.

TruckWEB said,
Nice concept, but it will require lots of free space and clear walls to be good. But that's innovation.

Unless you had some sort of camera-like device that could detect and map out the 3D layout of the space in front of it, and then render an environment against that layout to use as the projection.

But such a peripheral is surely hundreds of years away.

Joshie said,

Unless you had some sort of camera-like device that could detect and map out the 3D layout of the space in front of it, and then render an environment against that layout to use as the projection.

But such a peripheral is surely hundreds of years away.

Not really that far out. The software and hardware exists to be kludged together.

We already have software to do 3d maps of pictures, 360 degree cameras already exist. Just need someone to put it together.

TruckWEB said,
Nice concept, but it will require lots of free space and clear walls to be good. But that's innovation.

It really doesn't need the clear walls for peripheral vision augmentation.

The human mind and eyes would quickly erase out the objects and only use the information from the project rather quickly.

Anyone that has had a projection TV or worked with projection technology, it is amazing how messed up you structure where you put the image and how well the human mind compensates.

This is also not a 'primary' display, so it is just 'blurry' peripheral vision items. Right now everything in this area of your vision is blurred and also 'obstructed', you just don't pay attention.

The human eye itself is filled with blood vessels and other obstructions that you NO longer see, because your brain filters them out.

Work with people that have vision loss due to Macular Degeneration, depending on the severity, they can compensate for the 'blank spots', which is why a common 'test' for the condition is a page of horizontal and vertical lines and if your brain is newly 'compensating' which makes the lines curved.


Besides, you would be surprised how many gamers out there have dedicated spaces for their XBox or have it incorporated into a designed theater room. Even if it is a Den or 2nd Bedroom that is for gaming and movies. Not everyone games in their living room, especially if they have families and don't live in a basement.

the better twin said,
Yeah this wont happen for a good few years and who has a room with 4 clear walls. Cool concepts but thats all they seem to be atm.

Yeah, I'm not moving my Xbox to a small enclosed room. It's a cool concept and we'll see how they implement this, if they ever do.

the better twin said,
Yeah this wont happen for a good few years and who has a room with 4 clear walls. Cool concepts but thats all they seem to be atm.

I figured the idea was that the kinect could sense the 3d layout of the room, including objects, or non-flat walls, and compensate the project of the image so that the projected environment appears to enveloping the player - in a sense making it seem as though the objects in the rooms aren't there..

shao said,

I figured the idea was that the kinect could sense the 3d layout of the room, including objects, or non-flat walls, and compensate the project of the image so that the projected environment appears to enveloping the player - in a sense making it seem as though the objects in the rooms aren't there..


You're right on the ball here. This is exactly how it would work.

On the other hand, the path to a final product forks many times (some people call it' compromise', but in reality it's just the fact that no entity stands alone in the market). There are other console competitors, and the fact that, even though everyone is moving toward a two-screen gaming experience (iPad+iTV, WiiU, Xbox+WinRT, etc), Microsoft's push will face an onslaught in both the mindshare space (Gabe will spew whatever he possibly can to negatively impact MS's influence in the entertainment space, despite essentially working on the exact same sort of experience MS is building) and the legal space (if MS does it, sue!).

What's important is that the technology is being developed. Even if the touted project never comes along, the tools that can make it happen are being built and could do other, unexpected, really cool things in the future.