Kinect hack demonstrates 3D webcam

Kinect is quite young in the technology world and already developers are having their go at the device. Just three days after its release, an open source Kinect driver was forged. After that people began experimenting with the capabilities of this device, such as a method of interacting with photos.

Yet another Kinect concept comes to the scene, being demoed on YouTube. Oliver Kreylos has developed a C++ implementation of Kinect that allows for 3D reconstruction of a room using a single Kinect system. This works by taking advantage of the photographic video stream that is seen by the camera, then merging that data with the depth video stream. The depth camera will create points that the software can interpret to help create a 3D image that one can move around.

The resulting product is an awesome live 3D "model" of a room and everything inside of it. The question now is just how sophisticated can this system get? Now that essentially the difficult part of mapping the streams together is out of the way, the posibility of adding in more Kinect systems looks to be something that could very well happen. Using just an additional Kinect and an algorithm to mesh the four streams would help offset the shadow and create an even better image. The more cameras, the better the 3D image quality, although the processing power and meshing of the systems would get quite high quickly.

Kreylos also is looking to see how this technology could be applied to mix people with computer-generated imagry, something that is quite a step above from simple webcam effects. He says though that he still is working on that, but we can be sure that more will come from him in the realm of Kinect modding. Kreylos also demonstrates how objects can virtually be measured all using the software he has created.

Report a problem with article
Previous Story

Google Maps Navigation coming to Australian Android devices

Next Story

Review: Logitech K750 solar powered keyboard

22 Comments

Commenting is disabled on this article.

Very cool. The launch software seemed disappointing (menu navigation seems really wonky, speech recognition problems, laggy and rough gesture detection) but these hacks give me a lot of hope. The cool part is the hacks will serve as prior art, and a lot of these innovations will enter the public domain so many independent developers will be able to try out cool stuff without having to pay royalties, and we may now see a lot of great indie stuff that could lead up to amazing games that we never would have seen otherwise. Great job, hackers!

The Kinect code on the Xbox has already auto-updated a number of times to improve some of the issues you mention. Let's hope they continue to do that!

Halcyon-X12 said,
Very cool. The launch software seemed disappointing (menu navigation seems really wonky, speech recognition problems, laggy and rough gesture detection) but these hacks give me a lot of hope. The cool part is the hacks will serve as prior art, and a lot of these innovations will enter the public domain so many independent developers will be able to try out cool stuff without having to pay royalties, and we may now see a lot of great indie stuff that could lead up to amazing games that we never would have seen otherwise. Great job, hackers!

Remember that all of this is patented by Microsoft, hardware and software, and protected by the almighty DMCA....Microsoft had already demo'd some of this previously, so it is a good bet they will fight it very hard(right or wrong as the case may be).

MS didn't patent all applications of the technology because they haven't thought of every possibility. Drivers and software developed using clean-room methods are very common and perfectly legal. New and interesting applications are being discovered by those who are exploring the open drivers and creating their own software, and the ones that haven't been patented already won't be patentable but will be open to the public. Methods that are common across all implementations will cause any pertinent patents to become invalid, because obvious methods are not patentable. This will allow a host of independent development that will lead to unique and innovative results!

Edited by Halcyon-X12, Nov 15 2010, 8:49pm : clarification of legality

1. i hope Microsoft patented this technology!
2. i hope this gets integrated in windows "8"

these vidz are nice!

k33nz said,
1. i hope Microsoft patented this technology!
2. i hope this gets integrated in windows "8"

these vidz are nice!


facepalm.jpg

If I knew how to do this kind of things, I would already implemented it on a Website which runs a Metro UI... It will go perfect. Pity I don't have the sufficient knowledge to do that..

Titoist said,
wow... webcaming at a whole new level. Imagine the possibilities in porn!
With enough Kinect cameras, you could essentially allow the viewer to control where the camera is in not only porn but many other things (think 3D tours of houses or structures, etc). This would allow them to look at virtually any object they want to and even measure it (for a home buyer planning where they want to place furniture, for example).

But yes, as far as porn goes, you could shoot the scene once and allow the viewer to put the camera where ever they want in the scene based on their own preference. This would of course require constraints on camera locations or lots of kinect cameras in very awkward positions (such as under the performers). Thats assuming Kreylos, or someone else, gets this working with more than a single camera. As it stands, it's a good concept, but somewhat useless in practical application if you can only see the object in 3D space from a very limited side.

Though this would make it possible to "scan" an object into a 3D object you could import into a 3D program using only one Kinect camera by showing all angles of the object to the camera. This would pretty much stop the need for full 3D modeling. Need a computer desk in your game? Just move Kinect around a computer desk in your house and let it scan and recreate the object in 3D for you. Only minor touchup would be required for most objects. More complex objects could be done (like humans) but would still need rigged and all.