Microsoft is working on transforming touch experiences across all devices

The way we interact with our devices has changed in many ways over the last couple of decades, but there remains one constant that continues to define the user experience. While eye-tracking, voice control and various other interaction methods have gradually become more commonplace, the core of the experience remains based upon touch.

But while the quality of touch input support has improved greatly over the years, the feedback that we, as users, get from touching a screen has not advanced significantly on the devices that we own. Some devices offer haptic feedback, but the basic vibrations that they send to our fingertips as we touch a screen are still fairly rudimentary. 

This is why Microsoft wants to take touch to a whole new level, expanding the field of computer haptics to "fully engage the sense of touch in user-device interaction."

Hong Tan, senior researcher and manager of the Human Computer Interaction Group at Microsoft Research Asia, points out that despite the advancements in touch and haptics over the years, almost everything that we do relies on our eyes seeing the screen to be able to fully interpret our actions and the device's reactions to our inputs. "With sight alone, most people are perfectly fine interacting with computing devices today," she says. "But how much more efficiently, how much more enjoyably, can we interact with computers? How much more accessible can we make them?" 

Tan's team is exploring ways to redefine touch experiences, providing more 'relevant' feedback than a simple vibration, and even changing the way that the user interprets the 'texture' of the glass on a smartphone or tablet.

For example, Microsoft removed the standard glass front from a Nokia Lumia 520 and added piezoelectric actuators under the bezel. "Now, when you're typing," Tan explained, "the glass literally bends instantaneously - very small bending, but that's enough to tell your finger that it feels like a key-click."

Her team did the same with a Surface Touch Cover keyboard, allowing it to provide the same sensation of actually pushing a button, rather than simply tapping a flat surface. She says that this kind of haptic feedback allows users to type faster, and with fewer errors, compared with devices that lack the ability to provide such sensations to the user. 

In a further demo, Tan showed a Lumia 920 that had been integrated into a testing rig to showcase some much more sophisticated touch experiences, based on the same underlying technologies. On the checkerboard, the black squares feel 'sticky' when swiping your finger across it, whereas the white squares feel much smoother. 

In this example, an additional glass layer is placed on top of the device, hooked up to piezoelectric actuators. The glass is vibrated at very high frequency, but as you run your finger across it, the glass can trap a tiny layer of air between the surface and your skin, which changes the way that you perceive the texture of the glass itself, by increasing or reducing the amount of friction your skin encounters as it moves across the glass. 

A fourth demo showcases electrostatic haptics, which take advantage of the fluids in your fingertips. When your finger touches the screen, a conducting layer above the glass on the device can localise the electrostatic charge in a particular area on the screen, attracting the opposing charge of the fluids in your fingers. This creates a force that gently pulls the finger downwards, generating friction and again changing the way that the touch feels when touching or swiping across the screen. 

"The thing that's really, really cool," says Tan, of all the work that Microsoft is doing in this field, "is to take a smooth piece of glass but make it feel different - it's almost magic." 

Watch the full video about the work of Hong Tan and her team at Microsoft Research below.

At this stage, you may well be thinking this sounds absolutely awesome - or, perhaps, you may think it's all a bit lame. After all, how will any of this actually make any real-world difference when you come to actually use your device? 

Microsoft explains: "Imagine experiencing a clicking sensation when pressing an on-screen button, sensing the weight of folders when dragging and dropping, and perhaps even feeling the texture of a sweater for sale online." These are just examples, but they give a broad sense of how such a simple idea could translate into a transformative user experience - with more research, and plenty of imagination. 

These technologies also have enormous potential to make computing a much more engaging and empowering experience for those with poor eyesight, or none at all. In fact, Tan has been working closely with Microsoft's Accessibility teams, to get a better understanding of how blind people interact with computing devices, and with objects and spaces in the wider world. 

Exactly when we'll see these emerging technologies become broadly available remains ambiguous, for now - but we know that Microsoft has developed advanced prototypes for new touch experiences under its McLaren project. The '3D Touch' system implemented on that now-suspended Windows Phone handset was different to the haptics-based approach revealed here, but it does nonetheless indicate that Microsoft is seriously working to bring new touch experiences to market. 

When they finally arrive, they could well make interacting with our devices a whole lot more engaging and immersive. We can't wait. 

Source: Microsoft Research | images via Microsoft, except where indicated

Report a problem with article
Previous Story

Apple said to launch the iPhone 6 on September 9

Next Story

Microsoft turns up the profit dial on Azure by cutting discounts

39 Comments

Commenting is disabled on this article.

Lets say I'm shopping for a new shirt, would this technology allow me to feel what that shirt feels like? Can it go to that 'level' of feeling?

If so, that would be insanely cool. I'm very interested to see where this goes.

I really hope that Satya Nadella can start focusing on getting Microsoft Research experiments to become actual products. The whole Skype translation thing had me excited. Whatever happened to that, is it coming in Threshold or something?

Also, I believe Microsoft Research are not the only people experimenting with this kind of stuff. I'm looking forward to the day it becomes mainstream.

They just recently showed the Skype translation service live on stage at the WPC'14 keynote, so it is in the works as an update to Skype I believe. Hopefully a preview or a full release will be available soon.

That is just dandy. However, until Microsoft fixes the UI problem with some versions of Windows-8.x; what difference will it make. Until the OS is installed and used, who cares about the flash and falderol?

Its all about haptics! Its about time, their 15 year old sticks deliver more information. I'm glad it may get more than lip-service.

Dashel said,
Its all about haptics! Its about time, their 15 year old sticks deliver more information. I'm glad it may get more than lip-service.

I get excited over the idea of Braille being a viable option in the future for text and whatnot for smart phones/tablets =).

I remember when the iPhone was released, some people said that phones would always need real physical keyboards or they would fail. I had long discussions with people, possibly here, that phones without physical keyboards would be a big fail. My point is look how far we have come in so little time, how many phones have a physical keyboard today? Now look at what they are trying to do.

derekaw said,
I remember when the iPhone was released, some people said that phones would always need real physical keyboards or they would fail. I had long discussions with people, possibly here, that phones without physical keyboards would be a big fail. My point is look how far we have come in so little time, how many phones have a physical keyboard today? Now look at what they are trying to do.

I think what theyre trying to do here is a little bit different than just having a physical keyboard. they're trying to simulate tactility, as in, feel that there's something under your fingers whether it be dragging a folder, pressing a button, "feeling" a sweater.

derekaw said,
I remember when the iPhone was released, some people said that phones would always need real physical keyboards or they would fail. I had long discussions with people, possibly here, that phones without physical keyboards would be a big fail. My point is look how far we have come in so little time, how many phones have a physical keyboard today? Now look at what they are trying to do.

Doing away with the physical keyboard brought so many advantages in terms of physical appearance as well as cost that the trade-off was easily made. Doesn't take away that touch typing is still a tricky skill for many users. I'm not saying this tech will revolutionize touch devices but it could be a really welcome feature as long as it doent come with disadvantages regarding appearance and cost.

I'm thinking ahead to when the Borg Queen blows over the skin of my Galaxy S17 and asks "was that good for you"...

Some of the concepts are cool. Like feeling textures. Not sure how it would feel to feel the weight of folders tho. Either way, would be nice to test a working concept.

techbeck said,
haha, Apple will call it innovative and magical while Google will be blamed for using it to steal user info.

Well yeah... Google will be tracking your finger swiping with it to determine how you tap the screen and what mood you have or something ;).

techbeck said,
haha, Apple will call it innovative and magical while Google will be blamed for using it to steal user info.

i'm sure Microsoft has the patents....so they'll be making money regardless

rocksturdy said,

i'm sure Microsoft has the patents....so they'll be making money regardless

All depends...but we were just making fun of the 3 companies and having a little fun. :)

techbeck said,

All depends...but we were just making fun of the 3 companies and having a little fun. :)

I am just glad someone got my reference and joined in =).

Scabrat said,
Man, in 20 years, when Google or Apple make this technology and actually implement it, it will be so cool!

What does that says about Microsoft if they can't implement it themselves in 20 years time and have to wait for others to make it cool? That they are incompetence and as slow as they've always been?

cetla said,

What does that says about Microsoft if they can't implement it themselves in 20 years time and have to wait for others to make it cool? That they are incompetence and as slow as they've always been?

That you don't know a joke if it hit you square between the eyes?

Scabrat said,
Man, in 20 years, when Google or Apple make this technology and actually implement it, it will be so cool!

And no love for Samsung?.

rfirth said,

That you don't know a joke if it hit you square between the eyes?

It is a joke sure, but the sad thing about Microsoft is that someone else's joke is their reality.

Microsoft has invented most of modern tech themselves but never implemented it well. Tablet PC, smartphones, smartwatches, an so on. So what will make it different next time around? This is Microsoft we're talking about. People just don't give too much hope in anything they do.

cetla said,

It is a joke sure, but the sad thing about Microsoft is that someone else's joke is their reality.

Microsoft has invented most of modern tech themselves but never implemented it well. Tablet PC, smartphones, smartwatches, an so on. So what will make it different next time around? This is Microsoft we're talking about. People just don't give too much hope in anything they do.

They never implemented it well? The real problem is that they never implemented it at all. They always relied on hardware partners, which always let them down.

cetla said,

It is a joke sure, but the sad thing about Microsoft is that someone else's joke is their reality.

Microsoft has invented most of modern tech themselves but never implemented it well. Tablet PC, smartphones, smartwatches, an so on. So what will make it different next time around? This is Microsoft we're talking about. People just don't give too much hope in anything they do.


Mind you, smartwatches haven't exactly caught on yet, even now. The big question is, are they something that is worth becoming popular anyways? Time will tell, I guess, as will the outcome of this whole Android Wear deal.

cetla said,

What does that says about Microsoft if they can't implement it themselves in 20 years time and have to wait for others to make it cool? That they are incompetence and as slow as they've always been?

Then in the same position as Sony (outside of gaming), they had the 1st e-ink tablet, 3d tv where they can project two different 2d screens to different players, and host of other innovated tech that other companies manged to make successful after the fact.

dvb2000 said,
so, "back to the future" eh?

Just more evidence, people prefer keyboards than touchscreens.

I don't think that's what the evidence points to. I think most people agree that if they had to choose a touch screen OR a keyboard for their smart phone or tablet they would choose the touchscreen.

But, if you are saying they prefer a tactile touch experience to a non-tactile touch experience, then yes, I agree. More people want to feel they are pressing things or feel something on the screen. That could be the next push in smart phones/tablets =).