Microsoft Kinect meets the Minority Report thanks to MIT

Kinect Minority Report

Take the Kinect and drop it in the hands of the geeky, but exceptionally talented folks at MIT and the result is nothing short of impressive - Minority Report impressive. As reported by Kotaku, a group at MIT coded up some hand detection software capable of distinguishing your hand and fingers within a cloud of 60,000 points and at a rate of 30 FPS. Add in a simple but sweet graphical user interface, the open source robotics package 'ROS, and the libfreenect driver that lets the Kinect interface with Linux and you have the groundwork for a real-life Minority Report interface. 

The group at MIT put together a two-minute video (shown below) that showcases this novel interface. With this system, a test user is able to scan through an image gallery by moving their hand in a circular motion. The user is able to grab an image and bring it to the forefront where he can zoom or rotate it at will. When done with the image, the user merely pushes it off the screen. The system is able to pinpoint the fingers, which appear as five white dots on the screen and the palm which appears as a pink blob. Of course, real-life performance with this Kinect-based system is not as fluid as the movie, but it is an excellent start. 

The work was spearheaded by the Learning and Intelligent Systems (LIS) group and the Robot Locomotion group, both of which work out of the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT. They have provided the code behind this project, tutorials, and a mailing list if you are interested in learning more about how this system works.  

Report a problem with article
Previous Story

Dutch arrest 19 year old for DDOS attacks

Next Story

Gawker source code and database leaked

18 Comments

Commenting is disabled on this article.

As for the start in technology this really is impressive..

Don't be fooled we have seen similar devices in the past, playstation eye for example which was a massive hit for the learning difficulty community as helped with motor skills and hand to eye co-ordination.

However this Kinect device with the constant and almost global development means we can see some impressive technology enhancements arrive in a short period of time.

Big thumbs up to this kind of product!

Imagine Kinect 2.0 with its own processor built-in and higher res cameras capable of detecting much more detail. It'll come eventually, this is only the beginning.

quote "Of course, real-life performance with this Kinect-based system is not as fluid as the movie, but it is an excellent start" sorry guys an Australian company has beat you to it - patents office been busy lately. The fluid motion is much better than the movie and will include 3D modeling in mid air like a potter's wheel. No Kinect and only a small thin sticky disc on the hand.

What is impressive or the progression in the use of this technology is using the existing hardware and expanding the software detection concepts to accurately track finger points insetad of just hands.

(Something that could also be done in XBox 360 games if the developer has a bit more extra processing left over in the application to dedicate to the additional tracking.)

As the Kinect with the XBox works now is amazing, but due to the quick development timetable, most of the existing titles were designed on the most basic concepts of Kinect and graphically are on par with fast development titles, which makes a lot of the current game line ups look more like Wii titles instead of rich environments that normal XBox gamers are accustomed to seeing.

Give the Kinect time in the hands of more developers, especially now that it is out there in good numbers, and we will see some inspiring games in new ways to interact with Kinect that also get back to being visually immerisive in graphic quality.

Developers have a hard time wrapping around new gaming concepts when they are minor changes, and the Kinect is far from a small change. Just adjusting to online gaming was a major gasp for developers in the world that connects with others, and is still in its beginnings as well.

A whole paradigm of creativity is now being opened that is probably driving many designers as crazy at first as it opening the console to a new world of walking into a game.

Demonstrations like this and expansion of the technology is where the XBox developers are at as well, with, what can we do and the wows of how we can create something outside of our concept of traditional gaming.

But I do like the additional finger level tracking, I can see a developer running with this and doing more accurate virtual musical interfaces on the XBox or the PC that are not even games, but fun and productive.

I wonder how much more computing is being used for the additional tracking. The Kinect is sucking about 5% from one core on the tri-core XBox, and I wonder if something like this moves this up a bit to like 10 or 20% of a core or is intensive enough that it consumes a full core just for the tracking.

thenetavenger said,
What is impressive or the progression in the use of this technology is using the existing hardware and expanding the software detection concepts to accurately track finger points insetad of just hands.

(Something that could also be done in XBox 360 games if the developer has a bit more extra processing left over in the application to dedicate to the additional tracking.)

As the Kinect with the XBox works now is amazing, but due to the quick development timetable, most of the existing titles were designed on the most basic concepts of Kinect and graphically are on par with fast development titles, which makes a lot of the current game line ups look more like Wii titles instead of rich environments that normal XBox gamers are accustomed to seeing.

Give the Kinect time in the hands of more developers, especially now that it is out there in good numbers, and we will see some inspiring games in new ways to interact with Kinect that also get back to being visually immerisive in graphic quality.

Developers have a hard time wrapping around new gaming concepts when they are minor changes, and the Kinect is far from a small change. Just adjusting to online gaming was a major gasp for developers in the world that connects with others, and is still in its beginnings as well.

A whole paradigm of creativity is now being opened that is probably driving many designers as crazy at first as it opening the console to a new world of walking into a game.

Demonstrations like this and expansion of the technology is where the XBox developers are at as well, with, what can we do and the wows of how we can create something outside of our concept of traditional gaming.

But I do like the additional finger level tracking, I can see a developer running with this and doing more accurate virtual musical interfaces on the XBox or the PC that are not even games, but fun and productive.

I wonder how much more computing is being used for the additional tracking. The Kinect is sucking about 5% from one core on the tri-core XBox, and I wonder if something like this moves this up a bit to like 10 or 20% of a core or is intensive enough that it consumes a full core just for the tracking.

Some very well made points and interesting ideas

nice to see Kinect for random uses like this. sadly, it's main use for the XBox360 seems to be not all that useful at the moment.

Raa said,
Didn't I see this a week or two ago?

The video was posted up on the 7th, Kotaku wrote it up on the 8th. So yeah, it is 4-5 days old. Thought it would be OK for a weekend post, when news is slower.

Kelly Hodgkins said,

The video was posted up on the 7th, Kotaku wrote it up on the 8th. So yeah, it is 4-5 days old. Thought it would be OK for a weekend post, when news is slower.


Regardless of how "old" this might be, I feel that it's very newsworthy and is one of the most interesting things I've seen on Neowin in a while!

What is great is its price : this kind of 3D sensing technology was already available, but at a very high price (can be in the $10000 depending on the device). This will really help research on new, multi-modal interfaces.

Alladaskill said,
This is great, really loving what people are able to do with the Kinect Technology.

Indeed. And if Microsoft are sensible, they will encourage this kind of behavior. This has the potential to seriously boost the market for Kinect.