Last week, Microsoft revealed that HP and Intel have signed up as partners to create new devices powered by its Cortana digital assistant. That announcement followed the official unveiling of Harman Kardon's new Cortana-based smart speaker, known as the Invoke.
Cortana is now available across all major platforms, and on a wide range of devices, including PCs, phones, mixed reality headsets, and more; and auto manufacturers are also exploring integration of the assistant with their connected car software systems. But one developer has created a new hardware experience for Cortana, which brings the assistant much closer to its origins.
Cortana was inspired by the artificial intelligence character that appeared in the Halo gaming series as a hologram. Cortana's development as a virtual assistant on present-day devices has been limited to two dimensions, but developer Jarem Archer (also known by the handle 'unt1tled') has created a 3D version of Cortana in holographic form.
Archer described his creation as a "Pepper's ghost hologram that uses Windows 10's native Cortana experience". He explained:
It's all powered by a Windows 10 device with 4GB of RAM and a built-in Arduino used for controlling the platform lights. There's a portable USB monitor on the top that reflects on 3 panes of mirror glass. The enclosure itself is a custom design I modeled. I then 3D printed each part of the assembly in PLA. There's also an omnidirectional microphone that sits on top of the unit, and a decent mini speaker built into Cortana's pedestal on the base.
Archer provided further insights into the software setup that he created to run his Cortana hologram:
A Unity 3D app that presents and animates Cortana in three different camera angles and communicates to a second app: a proxy service that parses data going to and from the native Cortana web service. The proxy app renders the HTML from Cortana result queries which is then presented in the Unity app. It also features real-time face tracking with the front-facing camera which moves the rendered camera perspective relative to a single viewer's head position - this makes it look a bit more 3 dimensional when facing it. The app also controls an LED strip surrounding the edge of the platform.
He added that his wife had assisted in his development efforts by volunteering her services for motion capture, using two of Microsoft's Kinect sensors, to create animations for the Cortana hologram. These include "an array of screens that Cortana brings forth with her hand motions"...
...and Archer said that "each screen displays a unique mock UI loop and each loop plays on a random screen at start". He also has a few more ideas for adding to these animations, as well as expanding the capabilities of the holographic assistant itself, but hasn't yet had the time to fully develop them.
You can read more about Archer's efforts, including some additional photos of the hardware that he created, on his site, and a further overview is available in the video below:
13 Comments - Add comment