EU's landmark new AI law bans social scoring, limits biometric identification

UPDATE: May. 21, 2024, 11:37 a.m. UTC The AI Act has been signed by the presidents of the European Parliament and the Council of the EU. It will be published in the EU's Official Journal "in the coming days" and enter into force 20 days after that. Then, the AI Act will apply two years after its entry into force. Mashable's original article is as follows.

The European Union has adopted landmark new regulation that aims to protect its citizens from high-risk AI.

The AI Act, approved by the EU today, aims to "protect fundamental rights, democracy, the rule of law and environmental sustainability from high-risk AI, while boosting innovation and establishing Europe as a leader in the field," a press release said.

Specifically, the regulation aims to address risks created by AI applications, prohibit risky AI practices, set clear requirements for high-risk AI, and define the obligations, conformity measures, and enforcement pertaining those who deploy high-risk AI applications.

Applications which are banned by this new regulation include:

• "biometric categorisation systems based on sensitive characteristics and untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases,"

• "emotion recognition in the workplace and schools,"

• "social scoring,"

• "predictive policing" when based "solely on profiling a person or assessing their characteristics," and

• "AI that manipulates human behaviour or exploits people’s vulnerabilities."

There are exceptions for law enforcement, but some types of applications are still either banned or can only be used in very specific scenarios. For example, real-time biometric identification systems are allowed to be deployed only if "strict safeguards are met," including requirements for "specific prior judicial or administrative authorisation." In an example, such systems could be used in cases of missing persons or terrorist threats.

General purpose AI (GPAI) systems will have to meet the EU's transparency requirements, such as compliance with the EU's copyright law. Very powerful GPAI systems will have to meet additional requirements, including systemic risk assessment and incident reports. All artificial or manipulated images, audio, or video content (commonly known as deepfakes), will have to be clearly labeled as such.

Roberta Metsola, the President of the European Parliament, called the AI Act "trailblazing."

This Tweet is currently unavailable. It might be loading or has been removed.

"It means leadership, innovation & new avenues. But equally respect for fundamental rights," she tweeted.

European Commissioner for Internal Market, Thierry Breton, called the new regulations "the world's 1st comprehensive, binding rules for trusted AI," adding that the EU is "regulating as little as possible — but as much as needed!"

The AI Act is still subject to some final checks and needs to be formally endorsed by the European Council. It should come into force 20 days after being published in the EU's official Journal, which should happen fairly soon, and will be fully applicable two years after that — except in some more serious cases such as banned practices, in which case it will apply six months after entering into force.