Daydream Labs and Google Research show us how having a face in VR could make all the difference.
You know how much we love making mixed reality videos and sharing them with the world. It’s one of the best ways to show what a user in VR is actually experiencing and the environment they’re interacting with. Instead of just cutting between shots of the user in a headset and screen capture of headset footage, mixed reality beautifully overlays the real world with the virtual, giving you a window into that person’s immersive perspective.
Don’t get me wrong, setting up a mixed reality shot can be complicated and tedious at time (we’ll save our process for another time), but the final composite, mixed, merged or whatever you want to call it reality shot is usually well worth.
But no matter how compelling mixed reality is, there has always been one annoyance — you could never see the user’s eyes or face. You could see the person moving and interacting in VR, turning their head, but their face was always blocked by the headset — kinda unsettling.
Now thanks to some mixed reality magic being cooked up over at Daydream Labs and Google Research, we may have just seen the future of mixed reality — and it has your face all over it.
Tom Small, Head of VR at YouTube Spaces, shared a video on YouTube (now deleted but available on Google VR’s channel) demonstrating a unique way to add a 3D model of the users face over the headset in mixed reality. They’re calling it “headset removal”.
On the top is the traditional MR result where the face is hidden behind the headset. On the bottom is our result, which reveals the entire face and eyes for a more natural and engaging experience.
The process starts by capturing a digital scan of the user’s face. Sitting in front of a 3D camera, the user moves their head side to side and their eyes left to right. This allows for the capturing, processing, and building of a dynamic 3D model of their face with their eyes moving in different directions. Then thanks to a marker code placed on the front of the HTC Vive headset, the headset placement can be tracked and the 3D model added to the mixed reality footage. You can read more in depth on Google’s Research blog here.
Voila, headset removal with creepy human eyes and all.
Although its hard to say whether this headset removal in mixed reality is just as unsettling as the headset itself, it does signal a technique that could one day make watching others in VR as entertaining from the outside as is for the user in virtual reality. But for now, this is an ongoing collaboration between Google Research, Daydream Labs, and the YouTube team that we can’t wait to see developed further.
For check out more mixed reality videos, be sure to follow us on Instagram or Facebook.
Image Credit: Google VR/YouTube
UPDATE 2/21/16 9:38AM PST: The full video was deleted from YouTube shortly after we published this article.
UPDATE 2/21/16 4:40PM PST: Google VR has re-uploaded the video on their YouTube channel, the post has been updated.
