Microsoft HoloLens at Build 2015
Note: Microsoft didn't allow any photography inside the Build 2015 demo room. The images below were taken of a HoloLens inside glass outside the demo room.
HoloLens, Microsoft's AR viewer, feels like the future of computing.
The headgear I tried at Build 2015 is still described as "early development hardware," and it definitely felt like it. But the potential and how close HoloLens is to achieving it is remarkable.
The moment I tried on HoloLens, I thought, "This is like having a PC on my face." It's not that fully functional yet, but that's how the headgear and what you see and can do with it make you feel.
There were no Mars walks or Minecraft demos in the HoloLens session I attended. Instead, I became a developer for 90 minutes, crafting an application in Unity and adding layers of HoloLens functionality as I went. Every time a new function was added, like gesture controls and spatial sound, I got to see how it appeared in HoloLens. The session was intended to show how easy it is to develop for HoloLens, but it also demonstrated what users will experience once it's available.
How HoloLens fits
HoloLens is essentially two rings, a thicker one along the outside that contains all the innards and a slimmer one on the inside that wraps around your head. The inner ring has a roller in the back to tighten and loosen it, and it slides forward and backward so you can adjust how tight HoloLens wraps around your head. HoloLens isn't supposed to sit on your nose, but I found the rubber nose guard that comes with it inevitably fell on my nose no matter how many times I pried HoloLens forward. Thankfully, it's optional and comes off easily. HoloLens felt a lot better for me with it off.
I also had a hard time getting HoloLens to fit just right every time I put it on. I had to regularly retighten, resituate, and realign the headgear. When everything was synced up and it fit nicely, the holograms were in full view and it felt right. But if it was too tight, too high or too far forward, it hindered the experience. Standing in one spot and not moving too much created the best overall viewing experience. The adjustment issues cropped up especially when I was moving around, which is part of the fun when wearing HoloLens.
If you have short hair or it's pulled back, you might not have a much of a problem as someone with long, loose hair, like myself.
In general HoloLens is comfortable and fits snugly, but like all VR and AR headgear, it's frontloaded. You can't help but feel like there's a noticeable amount of weight near your forehead. It wasn't heavy, but it was significant enough. If Microsoft can balance out the design and somehow put more weight on the sides or back, it would likely help alleviate the front-heavy sensation.
There's a soft cushion on the part that touches your forehead that's appreciable. It's a small touch, but one that makes it easier to forget you're wearing the viewer and focus on the holograms in front of you.
HoloLens looks and feels like a premium peripheral. Nothing about it screams "cheap" in materials or design, and that will likely be reflected in the price. HoloLens looks like it belongs in the office and would blend well in any living room. It may need some extra protection in a construction zone, but it felt like it could be up to the task in grimier conditions.
Holographic viewing
The hologram that was part of my HoloLens experience consisted of two floating spheres, two yellow slides and some stacked blocks situated on a pad of paper. It was called Project Origami, so the materials were meant to look and sound like folded paper.
As I went along adding functions to the hologram (which turned into a holographic game), I added controls (gaze, gesture and voice), sound, spatial mapping and the ability to pick up, move and place the hologram around the room. Finally, an underworld was added to the hologram so that when the spheres rolled off, there was an explosion and they fell into a gaping hole that appeared in the floor. Inside the hole was an underworld, complete with rolling hills and cranes soaring underneath my feet.
The resolution of the holograms projected onto the real-life world around me was excellent. They were vibrant, sharp, and realistic. When I moved around them, the holographic shapes behaved like real world objects, so I could see their backsides or not see them at all if they became obstructed by other holograms. When the paper spheres rolled onto the floor, they rolled around just like real balls would, bouncing around objects and seemingly real enough to pick up. When I peered into the underworld that opened up on the floor, it was like I was looking into, as my Microsoft HoloLens "mentor" put it, a world I didn't even know was there the whole time.
But looking at holograms overlain onto the real world is only part of the HoloLens equation. Controlling the holograms is the other.
The HoloLens gaze controls were responsive and should be easy for any user to get the hang of. It's the other kinds of input where HoloLens has slightly more trouble.
While voice controls worked, there was a lag between giving them and the hologram executing them. I had to say, "Let it roll!" to roll my spheres down the slides, and there was a one second or so pause before they took a tumble. It wasn't major, but was enough to make me feel like I should repeat the command.
Gesture control was the hardest to get right, but once I figured out the best distance to hold my hand away from HoloLens and where I needed to place my finger in order for my pinch to register, the control worked better than expected. Instead of reaching out like you're going to touch the hologram, gesture works best if you're holding your hand comfortably in front of you.
With the gesture control, I was able to select the Projet Origami diagram, use my gaze to move it around the room, and then use another pinch to get the hologram to stay. This wasn't Minority Report-level swiping, but it was impressive nonetheless.
The HoloLens has spatial sound, which takes the experience to a whole new level. Ambient music played in the background, but when a sphere rolled, HoloLens projected the sound of crumbling paper. As I moved closer to the hologram, the sound got louder and faded as I moved away. It added another dimension to the HoloLens experience that made it that much more immersive.
Of course, as I was viewing the holograms, I was still able to see the real people and objects around me. The holograms weren't disrupted if someone happened to walk through my projection. My favorite part of this HoloLens demo was a wire frame mesh that appeared on real-world objects to show how HoloLens sees them. The mesh went over people around me, though I could still make out their features from behind the wire. It was an immersive AR moment: the people around me became part of the holograph themselves.
It would have been less distracting if the hologram disappeared when I talked to someone. From speaking with Microsoft personnel, it sounds like this would be possible on other applications, but not this Project Origami demonstration. It also felt weird to talk to someone with HoloLens on; it felt like I was wearing sunglasses inside.
There were glitches in the software, and it didn't work perfectly every time. That being said, the problems weren't horrendous nor did they significantly take away from my enjoyment of HoloLens.
Early verdict
HoloLens isn't perfect, either in software or hardware. But it's close, and when Microsoft does fix its issues, HoloLens is going to be an immersive entertainment tool that should find a place in the work world, too.
It was, in a word, delightful. Once more robust applications become available and more reliable functionality is achieved, it could also become very useful.
GDC and first hands on review
Update March 2015: Looks like gaming is definitely a go for the HoloLens. During GDC 2015, Head of Xbox Phil Spencer led a panel discussing the importance of games on the augmented reality device:
"We see this as a full Windows 10 device with holographic capability,"
Spencer also mentioned that the HoloLens APIs will be made available with Windows 10 gaming SDK.
Original review by Mary Branscombe continued below ...
The HoloLens that Microsoft will ship will be a Star Trek-style visor that wraps around your head but doesn't isolate you from the world, with the Intel SoC and custom Holographic Processing Unit built in. What I tried out in the labs hidden (rather cheekily) under the Microsoft visitor centre didn't look like that, but the engineers and developers who built it say the experience I had with the developer rig is essentially the same.
The HoloLens developer rig is built into a frame that slips onto your head and gets strapped into the right position; there's a sensor array over your eyes to track where you're looking and sensors on the frame to track how your head moves and the Holographic Processing Unit hangs around your neck on a strap (it's about the size and weight of a car radio). Someone needs to help you fit it and before you do that you have to get the distance between your pupils measured. And when you have it set, there's a power cord coming down from the ceiling rigged to move with you as you walk.
All of that will vanish into the final product, so what I tried is very obviously early hardware - like the custom developer kits Microsoft makes for early Xbox developers. Microsoft didn't let us photograph the developer rig, and you need a special camera to capture the HoloLens view so our images were supplied by Microsoft; they're a rather idealised version of the slightly grainier view I saw.
The HPU, as Microsoft calls it, isn't that much of a misnomer (less so than suggesting that a retina screen has the same resolution as the human retina, say). What you see isn't a holograph or a hologram; it's a projection - but it's being projected onto holographically printed lenses, which lets Microsoft produce very cheaply the extremely complex lenses that turn the projection into the 3D image you see.
The HPU turns the graphics into the right signal to project onto those lenses as well as processing information from the sensors that tell it where you're looking and how you're moving your head. It will speed up voice recognition and spatial sound processing, too.
That doesn't just let you see the digital world projected around you; it lets you see it on top of the real world. You can see the person standing next to you and talk to them, avoid walking into walls and chairs and even look at a computer screen, because HoloLens detects the edge and doesn't project over it so you don't need to keep taking it on and off as you work. You can take notes or answer email on a computer with a keyboard or a pen instead of trying to force that kind of close up work into the world of gestures and gaze.
If you wear glasses, this close-fitting headset isn't ideal. It's hard to make it fit comfortably - I found it either pushed my glasses down onto my nose or pressed them hard into my face, and I much preferred using it without my glasses on. As I'm short sighted, that made it harder to see detail. Talking to people outside Microsoft who've tried the actual HoloLens headset, it's light and comfortable to wear but the first version will probably still press on your glasses more than you'd like. And if you wear varifocals, you move your eyes automatically to look through the right part of your glasses for what you're focusing on; that can mean you look down at things that aren't in view for HoloLens (or for someone on a Skype call to your HoloLens) or look up and lose the HoloLens image.
The good news is that even if you're very sensitive to motion and prone to get VR sickness, or if you get headaches wearing 3D glasses, HoloLens is comfortable to use. I'm very prone to both of those and have problems with many other systems; after a brief moment the first two times I put it on when I could tell I was adjusting to what I was seeing, I had no problems at all with nausea, headaches or the other discomfort that can come when you trick your brain into thinking it's seeing something real.
The HoloLens projected screen moves as you move your head and you control apps either with voice commands or by using the equivalent of a mouse click - the air tap. You just hold your fist out in front of you where you can see it then raise and lower your finger. I didn't have to worry about getting it in the right place or moving it at the right speed; as long as I made sure my other fingers and thumb were out of the way, HoloLens got the gesture every time.
Digital reality
I tried three different applications with the HoloLens. I also got to watch several people using the Holo Studio 3D building tool, which has the most sophisticated controls, using a combination of gaze, gesture and voice commands to let you design objects you can see in the real world, so you know they're the size you want before you spend time and money 3D-printing them.
The most engaging was playing HoloBuilder; inspired by Minecraft and built with the help of the Minecraft team, this is a game that lets you build a digital landscape that exists in your physical space. Think the giant LEGO setup in the basement of Will Ferrell's house in The LEGO Movie, only invisible until you put HoloLens on, and built both on top of and underneath your furniture - and even extending under the floor and into the walls.
Using voice commands and the air tap gesture and my own real feet, I walked around a village, tickled a sheep to stop it falling off the table, dug through a (real) bench to make a hole, blew up some TNT to drop zombies into the lava pool that was in the chamber I'd dug through to, then blew a hole in the wall and lit a lamp to see the bats flying through the caverns in the wall.
Like Minecraft, half the fun is that your creations are obviously digital, but seeing them perched on real tables and benches was even more fun. This combination of virtual and physical worlds was delightful and immersive and shows the obvious gaming and entertainment potential here.
But the other two apps I tried were actually more impressive and certainly more useful. Making a Skype call from HoloLens is a good way to try out voice and gesture commands; you can look at the person you want to call in the address book - which is a grid of faces - then air tap to call them. The video call doesn't jump around the room if you move; it sits in one place unless you look too far away and then it moves back into view, or you can pin it in place.
The person you're calling doesn't need a HoloLens; they see in Skype what you're looking at and they can draw diagrams on the video that appear in your view. So if you're helping someone change a tyre or fix their dishwasher or fit a new light switch, you don't have to explain what they need to look for or pull out or unscrew - you can take a pen and show them.
This would fantastic for teaching and training, for remote support or for getting an expert opinion; imagine a remote handyman who could give customers advice (and charge them for it) or a repair service that always turns up with the right part because you've shown them what they're fixing in advance. It would also be a really interesting way to have a meeting where you're collaborating remotely on physical objects, not just the usual documents and presentations.
Remote working is something NASA has to do, but the OnSight system the Jet Propulsion Lab is building with Microsoft made me feel like I was walking on Mars. Some years ago I visited the Supervisualisation Lab at the university of San Diego, where they have a wall of screens that can show a life-size image of Mars using the photos sent back by the Mars Rover and I could look out at Mars like looking out of a window. HoloLens meant I could step out onto the surface of Mars and walk around, bending over to look at rocks, turning round to see the view and looking up to see the sky. This is where I most wanted to have peripheral vision because with HoloLens you only see what you're looking at; as you turn your head, the Rover suddenly comes into view and it's a rather large surprise.
OnSight isn't for digital tourism, delightful as that is; there are tools to tell the Rover where to go next to take photographs and samples - and the terrain that looks flat and easy to drive over on a PC screen is revealed as a treacherous series of slopes and ditches the Rover can easily fall down when you see it in 3D, so JPL scientists can look for an easy route before they air tap to give the Rover a target to photograph or burn with its laser.
They can also collaborate; other people in the landscape show up as stylised avatars, and a handy dotted line shows you where they're looking (HoloLens knows that, so it can show you), which avoids all the creepy 'uncanny valley' problems of realistic avatars. This is something we're going to have to work out social conventions for; when you say goodbye to someone miles away on the telephone it's easy to hang up, but when you can still see their avatar just turning away and ignoring them feels slightly rude. (Equally, watching someone use HoloLens is disturbing because they're turning and crouching and reaching out for things you can't see, and it's hard not to think they look strange.)
Walking around Mars with another scientist, or walking around an unfinished building and seeing where the walls will be, or being able to pull a virtual engine out of a physical car to see how it fits together so you can work out which screws to undo - there's a huge range of possibilities here, because this is far more than just a gaming system. And Microsoft is sensitive enough to privacy issues to have avoided the creepy feeling of using augmented reality to spy on the public world that Google Glass is so prone to. HoloLens doesn't just tag the physical world with information you can get other ways; it adds a realistic 3D digital world on top of (and underneath) it in a way that feels like magic and is delightful to use.
Early verdict
It's far too early to tell whether the HoloLens will be a success; only a select few have used the final design - instead of the early developer rig we tried out - so I can't comment first hand on how comfortable it is (with or without glasses).
I saw only four apps, one of which is more of a proof of concept (sorry Minecraft fans, HoloBuilder may not even ship). Price and battery life are both things I can only guess at. It's entirely possible that HoloLens may be more of a curiosity than a mainstream success. But if it takes off, and later models get smaller and lighter and less obtrusive, then Microsoft has just changed the world of computing again the way it did with Windows. HoloLens quickly feels natural and it's easy to see how useful it will be - and how much fun too.
from TechRadar: Technology reviews http://ift.tt/1IutOvt
