Grok ban: The nations considering blocking AI chatbot over nonconsensual sexual content

Grok, xAI's all-purpose helper, fact-checker, and content generator, is under close watch from international regulators who worry the chatbot is dangerous.

In early January, X users spotted that Grok was complying with requests to create sexualized depictions of minors in revealing clothing, a form of child sexual abuse material (CSAM). xAI staff said they were looking to strengthen the chatbot's guardrails.

But it's not a simple fix. Recent investigations by Reuters, The Atlantic, and Wired found widespread flaws in the model's safeguards, which have allowed for a proliferation of nonconsensual, sexual, and sometimes graphically violent content to be generated by Grok at the request of users. On X, investigators have uncovered a mass "undressing" problem, marked by numerous accounts of users who have had their publicly posted images edited by the integrated chatbot to don revealing attire. RAINN, the national anti-sexual assault nonprofit, has called this a form of AI or tech-enabled sexual abuse.

Other chatbots and AI-powered image generators have also come under scrutiny for failing to implement safeguards regarding sexual content, including Meta's AI companions.

Replying to user concerns on X, CEO Elon Musk stated that the company takes the generation of illegal content seriously, but argued that the legal ramifications should fall on the users. Musk later responded to comments from UK's leaders about a potential sitewide block, calling it a form of censorship and alleging that foreign governments are eager to silence free speech online.

Governments crack down on xAI

In response to slow movement from X's teams, countries have begun tamping down access to xAI's bot and standalone app, as several conduct investigations into Grok's safeguards, xAI's response, and the possibility that the company is violating various online safety laws.

The company has already faced several investigations before recent claims that Grok enabled CSAM.

In addition to ongoing investigation of X's content moderation and algorithms, members have pushed for the European Union to probe Grok for several alleged violations under its Digital Services Act, including potentially dangerous and "erratic" behavior. In recent weeks, the body ordered X to retain all data and internal documents regarding Grok as it expands its investigation.

Despite a commission spokesperson calling Grok's conduct "illegal," tech watchdogs are skeptical the body would invoke its measures to block the site completely.

The UK's communication regulator, Ofcom, also announced an official investigation into Grok, with a potential revenue fine of up to 10 percent (about $24 million) if it's found to have violated the Online Safety Act.

Organizations in the U.S. have called on federal agencies to investigate xAI, as well, citing the recently passed Take It Down Act, which criminalizes publishing or threatening to publish nonconsensual intimate imagery (NCII) and grants the Federal Trade Commission (FTC) authority to sue social media platforms that don't comply with take-down requests.

Other countries, meanwhile, are moving forward with outright bans until the company complies with investigations or overhauls Grok's safety structure.

Countries that have blocked Grok access

• Malaysia - Temporary suspension

• Indonesia - Temporary suspension

Regions actively investigating or threatening to suspend Grok

• India

• The European Union

• France

• The UK - Considering blocking services

• Brazil - Threatened to suspend Grok

• Australia - Already blocks social media access for users under 16

If you have had intimate images shared without your consent, call the Cyber Civil Rights Initiative’s 24/7 hotline at 844-878-2274 for free, confidential support. The CCRI website also includes helpful information as well as a list of international resources.