Facebook's AR glasses could give you super hearing

Augmented reality (AR) glasses promise a science fiction future, in which the visual reality around you is "augmented" with digital objects. But what if AR glasses could also give you hearing superpowers?

Facebook is working on something like that, and by the sound of it (sorry), the company isn't exaggerating when it says this technology will be akin to magic.

In a blog post Thursday, the audio team at Facebook Reality Labs Research shared some insights on what it's been up to lately. They're working with two primary goals in mind; one is to create virtual sounds that cannot be distinguished from reality, and the other, slightly more ambitious, is to "redefine human hearing."

To do the former, the researchers analyzed how your brain determines where a sound is coming from. That involves numerous factors, including the geometry of the room you're in, shape of your ears, different volume of sound in each ear, and the fact that one of your ears hears the sound slightly before the other.

And while near-perfect results still require some very expensive gear and a specially designed room, Facebook's researchers reached the point where people typically can't tell whether a sound in their headphones is real or virtual.

For the super-hearing bit, the team is leveraging tech that lets you hear better in noisy environments, augmenting desired sounds (like a person talking in a crowded room) and diminishing others (background noises).

Now, enhance that a bit, and imagine looking at someone and immediately hearing what she's saying far better than the other sounds in the room. Facebook's researchers plan to achieve this by capturing the sounds around you with multiple mics, and registering the pattern of your head and eye movements to figure out which sounds you're most interested in hearing.

All of this tech and more (read the entire post, it's very comprehensive) might make its way to Facebook's upcoming AR glasses, which Facebook has been working on since at least 2017. The glasses likely won't hit the market for a few more years, but when they do, they could have a far bigger impact than Google's short-lived Glass.