Google is training an AI to play video games — why this could be problematic

UPDATE: Mar. 15, 2024, 12:59 p.m. EDT This piece was updated to include additional information provided to Mashable by Google DeepMind.

Computer-controlled video game characters aren't anything new. Non-playable characters or NPCs are longtime staples of gaming. 

However, Google's AI research lab, DeepMind, has announced work on something distinct and new in computer-controlled gameplay: SIMA, which stands for Scalable Instructable Multiworld Agent.

This Tweet is currently unavailable. It might be loading or has been removed.

Unlike NPCs, Google DeepMind's AI-controlled gaming companion isn't programmed into a specific game to act a certain way or respond to certain human-provided actions. SIMA is, as DeepMind describes it, a "generalist AI agent for 3D virtual environments." 

SIMA is not an NPC

SIMA plays along with gamers, acting as a human player, by following verbal commands. As Google explains, "SIMA isn't trained to win a game." It's primed to work alongside the human player and perform actions based on natural language prompts.

This element is crucial to what makes SIMA unique. Again, SIMA isn't an NPC, designed specifically to a certain title, which is meant to provide a challenge to the player. SIMA acts as a fellow gamer trying to accomplish whatever the human player is telling it to do.

"SIMA needs only the images provided by the 3D environment and natural-language instructions given by the user," Google says. "With mouse and keyboard outputs, it is evaluated across 600 skills, spanning areas like navigation and object interaction - such as 'turn left' or 'chop down tree.'"

According to Google DeepMind, SIMA was trained and tested on nine different video game titles of varying gaming genres, including No Man’s Sky, Goat Simulator 3, Teardown, and Wobbly Life. The training does not require use of any sort of access to the games' source code or APIs. Google DeepMind says it focused on open-play games over narrative ones to facilitate SIMA's training in more general gaming skills.

There are some positives, and some very obvious negatives, with SIMA. An AI gaming companion could be nice when an individual wants to play a multiplayer game or an offline two-player story-mode without another human around to pick a controller.

But, the ideas driving SIMA's research could obviously be used in bad faith, too. Gamers could use SIMA for an unfair advantage in online gameplay. Games where players have to perform monotonous tasks for resources or complete quests to gain XP could obviously be manipulated by a human just telling an AI companion to do all of that work for them.

Google DeepMind reached out to Mashable to share that it has those same concerns regarding unintended uses of SIMA, as many on the team are gamers or former game developers themselves.

DeepMind reiterated that SIMA is a research project and video games are the sandbox that the team is using to test the AI agent's abilities. DeepMind currently has no plans to make SIMA widely available and says it will make sure it does not contribute to the bot problem in gaming.

As Google shared, its video gaming playing AI agent is not being prepped for public use at this time. For now, it's still in the midst of learning to play new video games.