Google wants to give you back your face when you're in VR

The blank, robotic VR masks seen in mixed reality videos could soon receive a facial fix that's pretty cool but also kind of creepy, just like any good advancement in technology should be.

In a new blog post on the Google Research Blog (hat tip to The Verge), a few researchers from the tech giant discuss the process of replacing the blocky VR headset with an image of the user's face for mixed reality videos -- videos that show both the VR user and the virtual reality around them.

The researchers hope they might eventually be able to extend this "headset removal" technique beyond mixed reality videos and into the VR world itself, increasing engagement and social interaction.

It sounds like a simple idea but it takes high-tech steps to get there. First, they take a 3D model of the user's face, including eye movements and blinks, which can be stored in a database for "gaze-dependent dynamic appearance."

That enables the team to project what looks like an active and reactive face onto the user later, not just an emotionally blank, unchanging face, which would be way creepier than just having the VR googles there to begin with.

A special external camera tracks the user during the VR session (shot in front of a green screen), enabling calibration between the aforementioned model of the user's face and the user's actual movements. The team can then render the portion of the face where the goggles are in the session video.

Eye-tracking technology on the VR headset gives the researchers a pretty good idea of what the user's eye movements and blinks are during use, enabling them to use the correct movements from the gaze database to give an approximate reflection of the user's actual movements.

The reason the goggles aren't completely removed from the final rendering is to give the user a "scuba mask effect" which researchers say keep viewers from falling into the uncanny valley, and being creeped out.

Check out the above video for a great run-through of the process which could be on its way to your average consumer at some point in the next few years.