Microsoft HoloLens: opening a portal to mixed reality

While consumers may be preoccupied with virtual reality and the industry is still exploring augmented reality, Microsoft has gone a different way: mixed reality.

The chief reason for this is HoloLens, the company's holographic computer system that it worked on in secret for years before unveiling it in 2015. The difference between virtual, augmented and mixed reality, Microsoft's Ben Reed explained, is not just a marketing term. Mixed reality can use elements of the real world and the digital world, but in a different way to augmented reality. "Most of the time these two universes do not collide," he said. "When we talk about mixed reality we're talking about the supersect that includes augmented reality and augmented virtuality. It depends on which end of the spectrum one comes. If one is starting in the digital world and bringing in real objects and environments, it's augmented virtuality."

The device is a head-mounted display packed with cameras, sensors and processors that sense the wearer’s movement and positions of objects around the room. The display is positioned in front of the wearer’s eyes, with speakers that “float” over the ears, built into the headband. It works with Windows 10, putting it on a familiar footing with users.

Microsoft chief executive Satya Nadella took over as head of Microsoft shortly before HoloLens was announced, and it's one of the company's projects that excites him.

“The fact that you can take your field of view and turn it into an infinite display, I think changes computing forever,” he said. “It changes large amounts of human activity like industrial design, architecture and education. When you’re not going to the computer but comp is there in front of your eyes all the time, [it]is a profound shift.”

Great potential

HoloLens is an ambitious project, but one with great potential. Although Microsoft showed off Minecraft and home entertainment applications at various demonstrations of HoloLens since its launch, HoloLens is not intended as a consumer product for now, Reed explained. Instead, Microsoft has been working with developers and partners to see what could be done with it.

What sets it apart from Oculus Rift, HTC Vive and other VR headsets is that it’s not about shutting reality out in favour of the virtual world, but rather about mixing the two worlds. The commercial possibilities are already intriguing. Elevator firm Thyssenkrupp, for example, has already begun deploying the headsets to workers, who can use HoloSkype to discuss potential maintenance works with remote experts. The company explained it cuts down on the time taken for each call; tricky repairs can be carried out with help and input from a second engineer, who can see exactly what the on-site worker sees thanks to the cameras mounted in the headset. They can then guide the on-site engineer through the process if needed.

Other companies that have got on board with HoloLens include Audi, which said it could use the device to improve service quality and cut maintenance costs. US firm Lowe's Home Improvement uses the devices to help customers make decisions on home projects.

Even Nasa has used HoloLens to create different experiences. For scientists, it created the OnSight project, which enables scientists to "work on Mars" from their offices, planning and – through the use of the Curiosity Mars Rover – conducting experiments. That led to "Destination: Mars", a visitor experience at the Kennedy Space Center, which is open to the public. It gives visitors the chance to walk around the Red Planet. Meanwhile, the Sidekick project, which is being conducted with the International Space Station, brings remote expert assistance to crews when they need it, and the ProtoSpace HoloLens application is designed to help build the next generation of spacecraft.

Anatomy lesson

On the educational side of things, there’s Case Western Reserve University, which facilitates interactive learning through HoloLens. One of the demonstrations of the device’s capabilities is an anatomy lesson that allows you to “see” the different layers of the human body and how its systems work, examining up close the human heart, all in 3D and all right in front of your eyes.

There's more on the way. Airbus and Saab are also set to use the platform in the future, and Reed said the real excitement came from seeing how the different partners decided to use HoloLens.

HoloLens may not have mass-market appeal for consumers just yet. The device is currently available in limited markets. It is already available in the US and Canada to developers and commercial partners, and Microsoft announced this week that the device would also be available for pre-order in Australia, New Zealand, France, Germany, Britain and Ireland. The devices themselves are expected to ship later this year.

But there may be a time, as the hardware develops further, that HoloLens and other devices like it may form a more important part of our lives than we think.