Microsoft's newest video tech can ID emotions, and see into your soul
What if you could understand everything about a video without ever having to watch it?
That may sound contradictory, but Microsoft has created a tool that can analyze videos and make their contents searchable, potentially saving you the trouble of watching the footage yourself.
Called Video Indexer, the software, which debuted onstage at Microsoft's Build developers conference, allows developers and publishers to quickly analyze huge amounts of video in surprisingly precise detail.
"The Video Indexer essentially allows a developer, or a publisher to upload a video... and then it will return a bunch of interesting insights about the video," Microsoft's Principal Product Manager for Cognitive Services, Irving Kwong explained to me the night before build in a preview briefing.
The Indexer can identify faces, recognize text that appears in a clip, come up with keywords, highlight text, and automatically translate speech into text captions in eight different languages. Not only that -- it's actually able to pick out feelings based on what's being said and the facial expressions of people in the video.
"Through the video, we use sentiment analysis to essentially look at faces and predict what the sentiment is at the time during the video," Kwong said.
In other words: You can run a clip through the Video Indexer and it will not only identify each individual and exactly when they appear, but it will guess their emotional state at each point in the video as well. What's more, analyzing a 45-minute video this way takes only about 4-5 minutes, according to Kwong.
Now that sounds like it's straight out of a sci-fi flick.
Of course, Microsoft isn't the only company working on this type of technology. Google has a similar tool that debuted earlier this year called the Cloud Video Intelligence API. Like Microsoft's Video Indexer, it also makes the contents of videos searchable, though it's limited to more general keywords.
It's also important to note that Video Indexer is not intended to be a consumer application -- at least not not right now. Microsoft is positioning this as a service for media organizations, advertisers retailers and even government agencies and law enforcement.
This is all either incredibly cool or the start of an Orwellian nightmare, depending on where you stand. Either way, though, it's clear our sci-fi future has finally arrived.
Topics Microsoft
Karissa was Mashable's Senior Tech Reporter, and is based in San Francisco. She covers social media platforms, Silicon Valley, and the many ways technology is changing our lives. Her work has also appeared in Wired, Macworld, Popular Mechanics, and The Wirecutter. In her free time, she enjoys snowboarding and watching too many cat videos on Instagram. Follow her on Twitter @karissabe.
