Computer mice to perform a disappearing act?

Have you ever wondered what technology might one day replace the Computer Mouse? If so, researchers at MIT may have your answer: Nothing! According to Wired.com, "In a magic trick that only geeks can pull off, researchers at MIT have found a method to let users click and scroll exactly the same way they would with a computer mouse, without the device actually being there."

They have appropriately named this new technology "Mouseless". It makes use of an infrared laser and camera to track your hand and finger movements that it then translates into computer commands. According to Microsoft User Experience Architect Daniel Wigdor, "Like many other projects in the past, including the Nintendo Power Glove and the Fingerworks iGesture Pad, this attempts to see how we can use new technology to control old technology. It’s just an intermediate step to where we want to be."

Despite the popularity of newer natural user interface technologies such as Voice Recognition and Multi-touch, the two-button mouse still remains the primary input method used on computers today. How much will this technology cost? According to Pranav Mistry, Project Lead for Mouseless, "A working prototype of the Mouseless system costs approximately $20 to build." Mistry previously developed the "Sixth Sense" project, a Minority Report-like wearable computer interface that lets users wave their hands in front of them in order to interact with maps and other virtual objects. 

Video Source: YouTube

Report a problem with article
Previous Story

Microsoft CEO: "Windows 7 slates by year-end"

Next Story

Impulse Top 10 for July 10th

110 Comments

Commenting is disabled on this article.

I find that device kind of lame... I mean, you're still using a mouse, in a sense..... Now, if Microsoft developed a mouse that worked by reading your thoughts, or tracking your eyes movements, and moving the cursor to the desired location, THEN i'd give them props.

I wonder how they are going to use it too work, kinect sort of technology to make it work, funny video though

Cute idea, but IMHO the tech is not going to make mice/touchpads/trackballs go away, though it may be a useful step on the way to gesture reading on hand-helds/cells. Practically speaking it can't be as precise, because when people move their hands it's not always about moving the mouse/trackball -- we twitch, flex, scratch etc. Nor does it save space compared to a std. note/net book touchpad or touchscreen. With mice avail. for <$10, it's not enough potential cost savings to matter to consumers.

OTOH monitoring/reading movements/gestures like in this project is cool when people can't use a mouse/trackball/touchpad etc... it could help some of the disabled, could be useful for those who don't yet know how to use a mouse, works when you're not stationary [i.e. using a handheld or playing some video games], & could work well in effect enlarging the the touch sensitive surface on something like a small cell.

Even if we could do away with the mouse, I think many people would still continue to use it as we like to have tactical feedback from a physical device that we control. It was a similar principle demonstrated in touchscreen phones - where the manufacturers purposefully included a vibrate feedback function when you pressed any key on the screen.

This reminds me of some projects I've seen at MSResearch. They developed a version of their Surface that didn't have a screen. You'd have a projector put the image on a table and you'd control it by touching the table.

Noticed the user had to keep his hand at the EXACT form/shape?

If he relaxes his hand a bit the positioning will be lost.

Gamers tend to relax their fingers every once in a while to loosen up... I'll stick with my wired mouse for now.

MrRoberts said,
Noticed the user had to keep his hand at the EXACT form/shape?

If he relaxes his hand a bit the positioning will be lost.

Gamers tend to relax their fingers every once in a while to loosen up... I'll stick with my wired mouse for now.

You do realize that this is no where close to market, right? When a tech is in academia it's mainly a proof of concept. Then they have to spend a couple years shopping the tech to industry to find a buyer or funding. Then a couple more years of development before the product would ever be released as a v1.

Picking on stuff like that with the tech in its current form is asinine.

I think this kind of technology will be useful for things such as laptop/netbook when you don't want to carry a mouse/turn on to look at something quickly/in an environment where a mouse is not required.

Or in home entertainment based computers where having a mouse/wireless keyboard could be not ideal giving how many people past by. I mean, you could select the movie you want to watch just by using your hands on the couch, it will be awesome. But this is not going to be good enough for modern technology. Lets face it, mouse aren't exactly the most annoying thing far as PC goes, people I know don't find mouses a nuisance even if they need to travel a lot. We are in an age where we want to do a lot in very little space, these smart phones are mini-computers in their own right the stuff they could be done on. This invisible mouse technology will be difficult to use if you want to access many buttons at once.

For me, a mouse that functions as a wireless or a wired mouse is the best for me. I like wireless, but if there is battery issue I could just switch, no problem.

Personally,
I'm not even going to say what I think of someone spending that much on a mouse, but not being squat for a gamer, I still don't want one!!

Not into wireless anything anyway.

Well its a given wired mouses are the best in terms of gaming. All else can use wireless. What I found with wireless was the hassle to change batteries whenever it runs out but that wasn't a bother unless I was gaming. This new mouseless mouse has lag that I can see when the guy moves his hand and the reaction time it takes for the infared/camera to processs it into the pc itself. Again this would be best used for anything else besides gaming as this would be far off worse than a wireless mouse in general. Maybe response time will be improved once its finally released or after a few patches but it looks pretty promising. Next stop, mind mouse control.

It's all preference, but IMO being a hard core gamer, a wired mouse is the best. I say this because a good wireless for FPS games is too expensive, sometimes heavier, and you have to deal with batteries.

The only way you can tell how good a mouse is, is to play a FPS since you have to be faster and more accurate (and have a better ping) than the person you are shooting. I've tried a few wireless (not the $150 ones) and just couldn't cut it for me in FPS games. Wireless was responsive, but that little big it couldn't do for me kept getting me killed. Switched the wired and went to the top of the list.

Good thing about this, everyone should go get a wireless mouse so I can destroy them with my wired. I remember winning a contest at a LAN gaming overnight thing because the other guy's batteries started tripping his mouse around when they got low.

If you ask me, it is more of a matter of opinion to be honest. Some people like wired mice and some like wireless mice. No harm in that!

I would go for a mouse pad that can track my movements. But in all honesty I still like wired mice for most daily tasks. I have a MS wireless mouse 5000 and its fits my hands the best and I really like it. Different strokes for different folks.

It would be good if this adapts into some sort of natal thing, control the pc with your hands etc but the mouse will stay for now, good to just have something to rest your hand on.

as most of those demo, they just "forgot" to mention all the problems.... a major one being no feedback = we need proprioceptive infos to know we have done something, and also misreading hand movements (aka imagine a sneeze translated to delete and confirm)...

Invisible keyboard existed since 2002 at least... and look how they are everywhere... </end sarcasm>

Ikshaar said,
as most of those demo, they just "forgot" to mention all the problems.... a major one being no feedback = we need proprioceptive infos to know we have done something, and also misreading hand movements (aka imagine a sneeze translated to delete and confirm)...

Invisible keyboard existed since 2002 at least... and look how they are everywhere... </end sarcasm>

Exactly. Somehow it seems developers are working on making more devices that are either not physical or have no physical feedback. I have no idea why because physical feedback is highly useful since it's easier to use something you can feel. Touchscreens with haptic feedback are coming eventually but unless they figure out how to give electric shocks to your fingertips or something I can not see much future for invisible mice.

Ridlas said,
No thank you. It looks painful to drag your hands on a table for long periods of time.

This seems to be what everyone is forgetting. There's way too much friction between your hand and the table.

This was what I reacted to as well, it's completely different from holding a mouse in your hand and moving it around - and it doesn't look comfortable at all. For everyday use, the ergonomics need to be majorly improved.

Totally remember that episode when I was younger. The lion intro is soo nostalgic too

And right after clicking play on the video, i was acting like i was using an invisible mouse

Hopefully this'll become a reality soon as this is pretty awesome.

Xilo said,
Can't have 5+ button mice with this.

No but why would you need 5 buttons? this is the "in-between" technology to bridge towards something more, but even this will be developed to act much more intelligently than a mouse, with gestures similar to multi touch pads... eventually moving us towards a system more inline with a two hand gesturing system I've seen showcased (however I have forgotten its name) - which has the ability to recognise each finger and consequently different figures can be used for different actions (simple example paint/erase/burn/dodge on 4 different figure in photo-editing)

lt8480 said,

No but why would you need 5 buttons? this is the "in-between" technology to bridge towards something more, but even this will be developed to act much more intelligently than a mouse, with gestures similar to multi touch pads... eventually moving us towards a system more inline with a two hand gesturing system I've seen showcased (however I have forgotten its name) - which has the ability to recognise each finger and consequently different figures can be used for different actions (simple example paint/erase/burn/dodge on 4 different figure in photo-editing)

I have a 7 Button mouse and its not enough.

you know

Fire/Aim/reload/Frag/Zoom/(scroll zoom in and out)Change wep +/- and a few otehr thing I use on the mouse that work better then trying to streech pink acrose buttons.

lt8480 said,

No but why would you need 5 buttons? this is the "in-between" technology to bridge towards something more, but even this will be developed to act much more intelligently than a mouse, with gestures similar to multi touch pads... eventually moving us towards a system more inline with a two hand gesturing system I've seen showcased (however I have forgotten its name) - which has the ability to recognise each finger and consequently different figures can be used for different actions (simple example paint/erase/burn/dodge on 4 different figure in photo-editing)


Right click, middle click, left click.

Also forward and back buttons which I use religiously in any game or program that supports it.

Xilo said,
Can't have 5+ button mice with this.

I would think this software will allow more than just 5. You have 5 fingers, and an infinite amount of motor skills that can be created into a macro style.

MadDoggyca said,

I have a 7 Button mouse and its not enough.

you know

Fire/Aim/reload/Frag/Zoom/(scroll zoom in and out)Change wep +/- and a few otehr thing I use on the mouse that work better then trying to streech pink acrose buttons.

Just by finger press combinations (not even stretch etc.) I think there would be 31 "press" combinations. (some of them are "awkward" to do however). But anyway you are thinking it has to be like a mouse still, an invisible keyboard would not appear to be sub standard because we don't have 80+ fingers.

Blacklights illuminate your fingertips.
infrared does too..
everyone knows camera tracking.

Problem is, you gotta act like you are using a mouse.. Im wondering when everythings goin to go vocal anyway... instead of message boards and typing - we just speak.. then use gestures to wade through and listen to what was said.

Ehh - stepping stone as said

Definitely a step in the right direction Funny how the mouse has just proven to be so difficult to replace over all these years...

Chicane-UK said,
Definitely a step in the right direction Funny how the mouse has just proven to be so difficult to replace over all these years...

"If it ain't broke, don't fix it"

Very cool and all but I have two problems with. First thing would be to try and not go outside the "layer" which as it's just behind the laptop screen would be very easy. Secondly damn that must be uncomfortable. My hand would prefer to rest on something.

Purify said,
Secondly damn that must be uncomfortable. My hand would prefer to rest on something.

I would believe that the user could simply rest their hand on the desk and then tap your finger up then down when you want to click. Because if not, you're right holding your hand up would be very tiring.

With this sort of technology why stick with a mouse??? expand get away from the whole "mouse" concept. I prefer Multitouch (my 32'' multitouch table speaks for itself) but I hope these guys don't become tethered to the whole mouse concept. this could be developed to be like a multi touch track pad, without the pad. much better. still they are only just getting started

I don't know if i like the idea of this for gaming though...perhaps i'm miles off the mark but nothing can beat the precision of the mouse when it comes to shooting someone in the head lol.

DKcomputers said,
I don't know if i like the idea of this for gaming though...perhaps i'm miles off the mark but nothing can beat the precision of the mouse when it comes to shooting someone in the head lol.

If this is true, I wonder why militaries seem to avoid mice for controlling predator UAV targeting or Remote Weapons System (RWS) interfaces - see the joystick behind the soldier's hand inside this Stryker with an RWS on top? http://en.wikipedia.org/wiki/File:Rws_stryker_interiorscreen.jpg

Even a portable tripod-mounted rifle RWS for mobile patrols used a thumbstick to aim. This is probably because, after investing millions in these weapons systems - they want to ruin the accuracy with an inferior input method.

Neb Okla said,

If this is true, I wonder why militaries seem to avoid mice for controlling predator UAV targeting or Remote Weapons System (RWS) interfaces - see the joystick behind the soldier's hand inside this Stryker with an RWS on top? http://en.wikipedia.org/wiki/File:Rws_stryker_interiorscreen.jpg

Even a portable tripod-mounted rifle RWS for mobile patrols used a thumbstick to aim. This is probably because, after investing millions in these weapons systems - they want to ruin the accuracy with an inferior input method.

I wanna see you pop a headshot in CSS with a joystick.

Twisp said,

I wanna see you pop a headshot in CSS with a joystick.

You can. If you use a professional gaming system like today's consoles where you have those professional tools like autoaim and autofire enabled. Because, no professional gamer is a true professional gamer without them, right?

God I hate these console FPS gamers that think they are good at FPS. But, I guess this is today's gaming generation.

Don't care, you'll never be as good as a true PC FPS gamer.

KavazovAngel said,

You can. If you use a professional gaming system like today's consoles where you have those professional tools like autoaim and autofire enabled. Because, no professional gamer is a true professional gamer without them, right?

God I hate these console FPS gamers that think they are good at FPS. But, I guess this is today's gaming generation.

Don't care, you'll never be as good as a true PC FPS gamer.

I don't get if you're flaming me or the top dude. lol

andrewbares said,
Google is set as their homepage in Internet Explorer? Booooo

wtf? Can't they have google as default when using IE? I too have that cause I'm used to search with google and most of the time I open a browser, it's for google. Why would I search with bing when I'm in IE and google firefox in firefox? Really, don't post comment if it don't have anything constructive.

Nodiaque said,

wtf? Can't they have google as default when using IE? I too have that cause I'm used to search with google and most of the time I open a browser, it's for google. Why would I search with bing when I'm in IE and google firefox in firefox? Really, don't post comment if it don't have anything constructive.

But they're using Windows 7 and even have a "I'm a PC" sticker on the laptop ((

And you're not being constructive either, commenting on an "unconstructive" comment.

andrewbares said,
But they're using Windows 7 and even have a "I'm a PC" sticker on the laptop ((

What does that have to do with anything? Google and Bing can both be used on PC obviously. I don't get it.

Neat but image tracking based solutions suffer from a number of issues. This and many other ones use software based tracking so not only do they require CPU power but they can also have considerable input lag. I guess that could change if hardware based tracking devices are developed but that would drive up the cost.

Another major set back is the lighting conditions. A lot of image based input tracking methods being tossed around use infrared images. The problem is there are tons to sources of infrared light coming from displays, lights and the Sun which can easily throw off the tracking. I think the best solution to that so far is using 3D based tracking which I think is less susceptible to lighting conditions but again that would also drive up the hardware costs.

Rob2687 said,
Neat but image tracking based solutions suffer from a number of issues. This and many other ones use software based tracking so not only do they require CPU power but they can also have considerable input lag. I guess that could change if hardware based tracking devices are developed but that would drive up the cost.

Another major set back is the lighting conditions. A lot of image based input tracking methods being tossed around use infrared images. The problem is there are tons to sources of infrared light coming from displays, lights and the Sun which can easily throw off the tracking. I think the best solution to that so far is using 3D based tracking which I think is less susceptible to lighting conditions but again that would also drive up the hardware costs.

Lefties need not apply? It will be a few years before I want multiple cameras tracking my hand movements, rather than just using a mouse.

It has nothing to do with privacy and everything to do with speed and efficiency. Those cameras are eating away at computer cycles, memory, and battery.

pickypg said,
Lefties need not apply? It will be a few years before I want multiple cameras tracking my hand movements,

specially when ur in a pron site right

DPyro said,
Do not want.

It's not something you'd be using all the time; just when you're out and about and you haven't got a full-sized physical mouse.

fourinjuly4ij said,
How do you click and drag?

Just like a trackpad on a laptop.

You double tap, leaving the second tap held down, and then move the file where you want to, and then release your finger to let go.

I doubt it would be as accurate nor as responsive as a normal mouse - two things that us gamers will never give up

Conjor said,
I doubt it would be as accurate nor as responsive as a normal mouse - two things that us gamers will never give up

Time will definitely tell. However, I don't really see myself giving up my mouse for a long time yet. In fact, I thought about getting my new dell laptop with multi touch, but then realized that I wouldn't really use it that often to be honest.

Neobond said,
I remember when the first wireless mice were useless with gaming, but now react as good as wired mice.

Gamers still predominantly use wired mice

Minimoose said,

Gamers still predominantly use wired mice


Thats because they are hard headed.
Wireless mouses are just as accurate as wired mouses nowadays.

alfaaqua said,

Thats because they are hard headed.
Wireless mouses are just as accurate as wired mouses nowadays.

1ms response time for wireless, right?

Not even in your dreams.

I'll agree that responsive wireless mice exist that are that accurate, but $130? Even most gamers would be like.... eh, no. My mouse is like $25/$30 and I game quite a bit.

KSib said,
I'll agree that responsive wireless mice exist that are that accurate, but $130? Even most gamers would be like.... eh, no. My mouse is like $25/$30 and I game quite a bit.

I didn't say it was affordable. I said it existed
My roommate got one of those. It's pure genius, but at that price, I am going to stay with my deathadder.

alfaaqua said,

Thats because they are hard headed.
Wireless mouses are just as accurate as wired mouses nowadays.

This

Gamers should be more concerned about their skill than the difference between 1 to 5 ms.

alfaaqua said,

Thats because they are hard headed.
Wireless mouses are just as accurate as wired mouses nowadays.
Fun when you're in the middle of a match and your battery dies!

dotf said,

Do you have one? I have the mamba, and I still prefer to use it in wired fashion for gaming. There is an initial hiccup whenever the sensor idles to sleep when it's wireless.


I have the mamba as well and I also mainly use it in wired mode, especially when gaming... not necessarily for performance, but mainly for the idle-thing and for battery (if it's almost dead and you're about to start a game, might as well plug it in).

hotdog963al said,
Fun when you're in the middle of a match and your battery dies!

A smart gamer makes sure he/she has decent batteries in it before starting a game, at least that's what I did

alfaaqua said,

Thats because they are hard headed.
Wireless mouses are just as accurate as wired mouses nowadays.

Only if you spend at least 100% more, so actually, they are being smart by opting for the wired mice

Minimoose said,

Only if you spend at least 100% more, so actually, they are being smart by opting for the wired mice

My logitech revolution is as smooth as a wired. I had 2 mouse connected, the revolution and my mx1000 and both have same response time in game. I'm always on my wireless, the wired is much more needed when battery died (about 1 time a weeks max and 5 min charge = more then 3 hours of gaming so). Honestly, if you really get bad score with a wireless mouse like this vs a wired one, it's more in the skill then the mouse itself cause 1 to 5 ms response time? I bet your monitor don't even have such a great response time.

z0phi3l said,

A smart gamer makes sure he/she has decent batteries in it before starting a game, at least that's what I did

A smart gamer makes sure not to buy a wireless mouse and hence doesn't have to worry about such stuff.

hotdog963al said,
Fun when you're in the middle of a match and your battery dies!

A gamer who's battery dies in the middle of the match, sucks there is no difference between if he had a wireless one or a wired on.

Nodiaque said,

My logitech revolution is as smooth as a wired. I had 2 mouse connected, the revolution and my mx1000 and both have same response time in game. I'm always on my wireless, the wired is much more needed when battery died (about 1 time a weeks max and 5 min charge = more then 3 hours of gaming so). Honestly, if you really get bad score with a wireless mouse like this vs a wired one, it's more in the skill then the mouse itself cause 1 to 5 ms response time? I bet your monitor don't even have such a great response time.

I'm not even talking about myself, so why would my monitor make a difference.

Nodiaque said,

My logitech revolution is as smooth as a wired. I had 2 mouse connected, the revolution and my mx1000 and both have same response time in game. I'm always on my wireless, the wired is much more needed when battery died (about 1 time a weeks max and 5 min charge = more then 3 hours of gaming so). Honestly, if you really get bad score with a wireless mouse like this vs a wired one, it's more in the skill then the mouse itself cause 1 to 5 ms response time? I bet your monitor don't even have such a great response time.

I was talking more about the quality of the sensor and the tracking in the mice. I have had bad experiences with laser mice, and wireless ones haven't been able to track well at the low sensitivity I use (although this depends on the mouse mat)

KavazovAngel said,

1ms response time for wireless, right?

Not even in your dreams.

1ms response time for your hand, right?

Not even in your dreams.

tomjol said,

1ms response time for your hand, right?

Not even in your dreams.

Your hand doesn't need a response time, what matters is the delay between your movement and the cursor.

tomjol said,

1ms response time for your hand, right?

Not even in your dreams.

Stick with your xbox controller okay?

Leave me to have fun with my Razer.

KavazovAngel said,

Stick with your xbox controller okay?

Leave me to have fun with my Razer.

I'll leave the Xbox controller gathering dust, you having fun with your Razer, and go outside and enjoy that thing most of us call 'summer'.

winlonghorn said,

Yes, using Tom and Jerry was great! lol.

Best tech video EVER.
So damn fresh and innovative, i can't say how awesome tom & jerry are.