Kinect has problems recognizing dark-skinned users [Update]

This morning Kinect was launched in North America to millions of eager customers, and as reviews are still rolling in on the device, another weird "review" states that Kinect can't properly recognize dark-skinned users, according to Gamespot.

According to the report, Kinect had trouble with the facial recognition with two out of three dark-skinned employees. The system was unable to properly track two of the employees faces, but didn't have any trouble tracking their body movements. Even with repeated calibration attempts, Kinect was unable to properly identity one of the dark-skinned employees.

This doesn't prevent dark-skinned users from playing the system and enjoying the gameplay, but some of these users may need to sign in manually each time to get Kinect started, rather than relying on facial recognition to sign them in.

This wouldn't be the first time that a device had troubles detecting dark-skinned faces, such as HP's webcam had trouble detecting non-caucasion faces. The software algorithm had issues tracking dark-skinned faces, much like the Kinect does.

This minor annoyance with the add-on shouldn't deter people from purchasing the new device, but be cautioned that not all of the features will be available to dark-skinned people. Hopefully Microsoft will release an update or quick fix for the issue in the near future. Don't forget to check out how Kinect works here.

Update: Consumer Reports claims that the initial reports by GameSpot was untrue, and the issue deals with poor lighting. However, until further tests can be performed, it's hard to believe which story is true.

Report a problem with article
Previous Story

How Kinect Works

Next Story

Daily Gaming: November 4, 2010

104 Comments

Commenting is disabled on this article.

And I thought people knew this,Melanin absorbs light so what do you expect.
Pluses: No skin cancer
Age at least 20 years slower then everyone else
Adapts to changing UV radiation conditions, Like the not too distant future when global
warming and UV levels get worse.

Minuses: Poor Kinect facial tracking

Other pluses and minuses
http://en.wikipedia.org/wiki/Melanin


Julius Caro said,
I wonder what features the thing relies on for facial recognition

I'd guess simple things like the distance of the eyes from each other and the space between the mouth and the nose.

Pupik said,
I'd guess simple things like the distance of the eyes from each other and the space between the mouth and the nose.

Microsoft Research Asia have been doing a lot of work in this space for a lot of years. I would imagine they would have borrowed from those learnings. As a result, I believe your assumption to be an oversimplification of the solution.

They should release two Kinect versions for black people and white people with different algorithms and a different box color (black and white).

Well jeez.. you'd have thought they'd have tested it on people OTHER than white people This is potentially a major embarrassment..

************THE SOLUTION***********
Every Kinect unit should be sold with a a can of white spray paint. Problem solved.

This is an issue in general photography with darker subjects. Additional lighting is usually need to capture fetures of a subject with a dark complextion.

One solution I have read about is to use IR LEDs. The camera can use this invisiable light to provide additional light (in composite) to capture features in low light or otherwise hard to distinguish subjects. I'm not sure what the situation or solution would be on Kinect. I'm sure it would add additional cost, and I'm sure they are already biting a big one with the combined price of the state of the art hardware and software.

I have seen webcams implement the IR LEDa and sensor, but have not seen any particular instances of their use in regard to computer vision with facial recognition. I imagine it has been done and if it improves recognition (especially for this senario) it is something MS should have considered in their tests.

Shame it has to come to light like this. It's a little humorous (a head shaker), but still pretty sensative topic. I'm assuming most people are laughing at MS... I hope...

oh, according to the other neowin kinect news, kinect does have an "invisible light source". so there. shouldn't be THAT much of a problem if they use infrarred, though not sure if they use that for the face recognition part

i think they did this on purpose
remember when microsoft accidentely put a black head on a white persons body on one of their websites

sweet5695 said,
i think they did this on purpose
remember when microsoft accidentely put a black head on a white persons body on one of their websites

That was not MS, it was a design agency decision.

cameras generally will have an easier time picking up lighter colors better than darker colors. having a well lit area will help this.

when it says "dark-skinned", is that like Wesley Snipes dark or Will Smith dark?

Interesting, though the impact of this type of racial bias (and gender bias) has been known in other fields than game-development for a long time.

tensegrity said,
Interesting, though the impact of this type of racial bias (and gender bias) has been known in other fields than game-development for a long time.
Gender bias? I didn't realise the two sexes reflected different amounts of light. (Well, there is makeup, I suppose).

It is a bit distressing that they didn't bother testing dark-skinned people or ensuring diversity in their beta testing programs.

Fred Derf said,
It is a bit distressing that they didn't bother testing dark-skinned people or ensuring diversity in their beta testing programs.

It's only the facial recognition and its spotty. I don't think dark skinned people who are rational will have an issue. If it can be fixed, I'm sure it will. It's going to be mostly a lighting issue. A dark skinned person expecting it to recognize them in a dark room is not going to be offended, they are going to be stupid. It would be interesting to see a white person in an all white room with white clothes. Not very common situation. Dark skinned person in poor room lighting very common.

Fred Derf said,
It is a bit distressing that they didn't bother testing dark-skinned people or ensuring diversity in their beta testing programs.

They quite obviously tested out dark-skinned people - however it is quite simply a technological limitation. You'll be hard pressed to find ANY facial recognition system that doesn't become less accurate with dark skinned people.

Critical Error said,
So, their E3 video was fake? http://www.youtube.com/watch?v=6KKgtN4T8V8

It can track and follow them fine, meaning they can play games the same as anyone else. It just can't recognize or distinguish their facial features as easily as lighter skinned people, which is basically used for... just the automatic login? That's the only thing effected here.

~Johnny said,

It can track and follow them fine, meaning they can play games the same as anyone else. It just can't recognize or distinguish their facial features

*cringe*
So the Kinect can see black people, it just can't tell them apart... urghhh... I've got the feeling that conventional media is gonna have a field day with this.

*Der Herp*! M$ is so R4cist! They hate teh black People!

That is seriously what some of your guys' comments sound like. This is old news, its harder for motion tech to sense darker skin. So lets please let the dumb racist accusations rest.

Well this explains the Microsoft picture where they altered out the black guy at the conference table....... j/k

neufuse said,
Well this explains the Microsoft picture where they altered out the black guy at the conference table....... j/k

Of courrse the expected "racist" tags had tp be used. Grow up jerks!! Are you interns for the new sensational nwws format? Should this have been better tested by MS ? Yes.!! MS is not racist when it comes to selling product. Long as rhe money is in the bank!

Meconio said,
Somebody at MS tests lab is sooo gonna get fired.

Why would they? They cannot control that the color white reflects light, allowing the camera to capture more light and details. While the color black absorbs light which reduces the amount of light reflected back to the camera. It is Physics not an actual mistake. Microsoft will probably just tweak their algorithms to make it easier for the software to detect darker colored people.

Kutster said,

Why would they?


Dude I was just kidding, this at most deserves a "What happened there?" talk, but you have to acknowledge that this is the kind of thing that you should have come out at the test lab before deploying it, but it's obviously a minor bug that just need a software tweak...I just find the whole thing kind of funny.

Hmmm.... Neowin, something's fishy on this news page,,,

Warning: Visiting this site may harm your computer!
The website at http://www.neowin.net contains elements from the site images.planetamd64.com, which appears to host malware - software that can hurt your computer or otherwise operate without your consent. Just visiting a site that contains malware can infect your computer.
For detailed information about the problems with these elements, visit the Google Safe Browsing diagnostic page for images.planetamd64.com.
Learn more about how to protect yourself from harmful software online.
I understand that visiting this site may harm my computer.

I got the above from Google Chrome when clicking on this story.

TCLN Ryster said,
Hmmm.... Neowin, something's fishy on this news page,,,
I got the above from Google Chrome when clicking on this story.

I went through the same thing too.

As a dark skinned man, I can confirm it. Last night (midnight release) when I setup, it worked fine but this morning I had problems recognizing me. I guess I need to tune it in different lighting conditions. No biggie though. Still awesome and happy with my purchase.

Making funny faces when the camera takes a pic in Kinect Adventures is fun.
http://sphotos.ak.fbcdn.net/hp...4278_45560660_6718214_n.jpg

Zedox said,
As a dark skinned man, I can confirm it. Last night (midnight release) when I setup, it worked fine but this morning I had problems recognizing me. I guess I need to tune it in different lighting conditions. No biggie though. Still awesome and happy with my purchase.

Making funny faces when the camera takes a pic in Kinect Adventures is fun.
http://sphotos.ak.fbcdn.net/hp...4278_45560660_6718214_n.jpg

Yeah, it's a problem. Cuz you're not that dark. Luckily, it's just the facial recognition part. Most people will start the thing normally I would guess. I think a little more light would help based on the pic.

I'm gettin' one.

Zedox said,
As a dark skinned man, I can confirm it. Last night (midnight release) when I setup, it worked fine but this morning I had problems recognizing me. I guess I need to tune it in different lighting conditions. No biggie though. Still awesome and happy with my purchase.

Making funny faces when the camera takes a pic in Kinect Adventures is fun.
http://sphotos.ak.fbcdn.net/hp...4278_45560660_6718214_n.jpg

You're not that dark, so it is a problem then. I'm still getting one. Since it only affects login, no biggie, but they shoud fix or make lighting recommendations. I'm getting one.

Zedox said,
As a dark skinned man, I can confirm it. Last night (midnight release) when I setup, it worked fine but this morning I had problems recognizing me. I guess I need to tune it in different lighting conditions. No biggie though. Still awesome and happy with my purchase.

Making funny faces when the camera takes a pic in Kinect Adventures is fun.
http://sphotos.ak.fbcdn.net/hp...4278_45560660_6718214_n.jpg


Dayam, THAT is too dark for kinect?

Looks like albinos will be the only ones having fun?

yardman said,
Oh please i smell FUD.

No, it is true. HP also had a problem like this. There is even a YouTube video proving it.

The only way I can really thing to fix the accuracy of facial recognition for dark-skinned users is to shine a bright light, like those LED lights used for cell phone flashes when detecting the face. The problem is facial detail is difficult to see on a dark-skinned person at a distance unless you have enough light to light up their face enough to more easily see the detail in their face.

kInG aLeXo said,
Maybe they need to add better lens (which would raise the cost of the device a lot) ? or get better light in the room ?

I'm guessing it needs better lighting to recognise people with really dark skin - after all the facial recognition part is coming from the standard RGB camera, and it'll just be harder to make out their features.

How dark are we talking here? it just seems to say dark skinned.
Are we talking a crow in the middle of the night dark? or slightly darker than most light-phobic caucasian gamers (like me, the light-phobic that is)?
This article seem light on information and I can't watch the video right now as I have internet problems

Teebor said,
How dark are we talking here? it just seems to say dark skinned.
Are we talking a crow in the middle of the night dark? or slightly darker than most light-phobic caucasian gamers (like me, the light-phobic that is)?
This article seem light on information and I can't watch the video right now as I have internet problems

I think they are just referring to black people. Probably they thought dark-skinned sounds more politically incorrect that saying black.

AtriusNY said,

I think they are just referring to black people. Probably they thought dark-skinned sounds more politically incorrect that saying black.

Well, you know, Indians and other middle easterners are actually darker than most black Americans. So, dark-skinned is appropriate in our melting pot of a country. And don't forget Hispanics. So, "how" dark are they talking about is a good question.

I mean, some white people tan to the point of looking Hispanic, will they have a problem?

AtriusNY said,

I think they are just referring to black people. Probably they thought dark-skinned sounds more politically incorrect that saying black.

I hope not. There's nothing wrong with calling a black person black.

Hardcore Til I Die said,

There's nothing wrong with calling a black person black.

Correct, and African American doesn't work too well in Europe either.

ILikeTobacco said,

hopefully not. we have enough douchebags on xbox live as it is

ILikeTobacco said,

hopefully not. we have enough douchebags on xbox live as it is

Bazinga^2

AtriusNY said,

I think they are just referring to black people. Probably they thought dark-skinned sounds more politically incorrect that saying black.


Well, it at least sounds a bit more ridiculous.

Or ... OR ... dark skinned is actually the accurate portrayal of the issue. It's about darkness, not colour. Referring to a 'racial' colour doesn't make sense, when it can affect more than those connected to that race.

Kirkburn said,
Or ... OR ... dark skinned is actually the accurate portrayal of the issue. It's about darkness, not colour. Referring to a 'racial' colour doesn't make sense, when it can affect more than those connected to that race.
Who gives a flying ****!

littleneutrino said,
Reminds me of the Motion Detectors from "Better of Ted"
lol, bundle a free white guy with every Kinect

littleneutrino said,
Reminds me of the Motion Detectors from "Better of Ted"

Haha, I'm glad somebody else saw that series.

Webbster said,
Were the last couple of episodes of season 2 ever aired??

Aired in Australia. You can find them online, if not, I can post a torrent file somewhere for anyone who wants it.

I also remember the similar issue with HP face-detecting webcams, it's just a technical issue rather than an intentionally racist one I guess.

jamesyfx said,
I also remember the similar issue with HP face-detecting webcams, it's just a technical issue rather than an intentionally racist one I guess.

its not "software algorithm" is the cheap hardware used in kinect and poor light conditions, if they would use the console in brite lit room it would work better. Just like Digital camera often have trouble taking pictures of dark skinned indivuals.

jamesyfx said,
I also remember the similar issue with HP face-detecting webcams, it's just a technical issue rather than an intentionally racist one I guess.

Q: But what would Steve Jobs say if this is his product?

jamesyfx said,
I also remember the similar issue with HP face-detecting webcams, it's just a technical issue rather than an intentionally racist one I guess.

does that mean HP had something to do with the development of kinect

dotf said,
Being American, he would most likely say you are the wrong color.

Being English, he would know that colour and color are both pronounced the same and SAY the same.