Google's tricking us with human-sounding AI, and we're scared

It was the "mm-hmm" heard round the internet.

At Tuesday's Google I/O keynote talk in Mountain View, California, CEO Sundar Pichai introduced the gathered developers to Google Duplex — a version of Google Assistant that will make actual phone calls for you and, in the process, trick the person on the other end of the line into thinking she's talking to a real-life human. Yes, verbal mm-hmm ticks included.

While currently limited to certain tasks like booking appointments, we shouldn't expect AIs like Duplex to remain that way. As Google Principal Scientist Greg Corrado made clear at a Wednesday keynote addressing the future of artificial intelligence, what we saw Tuesday is only the beginning.

Corrado explained that over the next decade he expects to see the "development of artificial emotional intelligence” that will allow "products to actually have more natural and much more fluid human interactions.”

Even more natural than the above phone call. All of this should probably worry you.

It clearly didn't bother those at I/O, though; everyone was either too wowed, dumbfounded, or just plain impressed to suppress a collective round of laughter and applause when the AI dropped a little "mm-hmm" mid-conversation, just like an actual person would.

Don't get us wrong — it is impressive, and there are clearly potential benefits for those with speech-related disabilities. Even so, perhaps a gasp in response would have been more appropriate.

While it's too early to know what the finished product will look like (Duplex is being made available "as an experiment" in the coming weeks), the demo provided by Google was supposedly a recording of a real phone call. In the portion of that call played to the I/O audience, the AI made no disclosure to the person on the other end of the phone as to its silicon origins.

This Tweet is currently unavailable. It might be loading or has been removed.

At its core, Google Duplex is based on deception. According to a Google blogpost, "[the] system also sounds more natural thanks to the incorporation of speech disfluencies (e.g. 'hmm's and 'uh's)" which mimic "what people often do when they are gathering their thoughts."

Why does any of this matter? What conceivable problem could there be with AI that can carry out conversations with people so convincingly that those people believe they're talking to another person?

Ah, robocalls. According to The New York Times, there were 3.4 billion automated calls made in April of this year alone. Data provided to the Times by YouMail, a robocall blocking service, noted that in March there were almost 123 million automated calls pushing a zero-interest-rate scam of some sort. How much more difficult it will become for the average consumer to defend against this sort of fraud when the AI on the other end of the call can persuasively talk you into whatever its creators have programmed it to do?

That doesn't even touch on the basic question of whether or not we have a right to know if we're talking with a human or an AI. Google, for its part, belatedly admitted this dilemma in a conversation with The Verge. A spokesperson told the publication that — and this is The Verge's language here — "it definitely believes it has a responsibility to inform individuals."

While that's a start, we need assurances more specific and detailed than those provided by a spokesperson talking on background to a tech publication. Because as products like Google Duplex become mainstream, and so-called artificial emotional intelligence advances closer to a reality, we may all soon be forced to puzzle out whether our conversations are happening with a human — or a machine.

If we're not careful, the answer will be a resounding mm-hmm.

This post has been updated to include information a Google spokesperson told The Verge.