Meta's crackdown on adult content fails to stop AI nudify apps from flourishing

Adult content, including AI-generated nudes, remains banned on Facebook and Instagram, despite recent policy changes at Meta.

However, that hasn't stopped "nudify" apps — platforms that create nonconsensual AI-generated nudes of celebrities and influencers — from advertising on Meta's platforms. And those ads have been extremely successful for at least one service called Crush AI.

According to a new report from 404Media, as originally reported in Alexios Mantzarlis' Faked Up newsletter, the Crush AI modifier app receives the vast majority of its traffic from Meta platforms such as Facebook and Instagram. According to Similarweb's data, last month Crush AI received more than a quarter of a million visits to its service — and roughly 90 percent of that traffic was from Meta's platforms.

Crush AI makes it quite clear what the purpose of its service is in its advertisements. The company's ads include photos of real-life individuals like Instagram influencer and model Mikayla Demaiter and OnlyFans creator Sophie Rain. The ads boast that users can "upload a photo" and "erase anyone's clothes."

While Meta has attempted to crack down on these ads, Crush AI has discovered a very simple workaround. Crush AI creates fake profile pages using AI-generated profile pictures and runs new ads using those pages. The ads promote different domain names that then redirect to Crush AI when clicked.

Mashable has previously reported on other deepfakes and AI platforms that utilize Meta's advertising platform to generate traffic and revenue from AI-generated adult content. These ads are strictly against Meta's policies, and the company often removes them once it becomes aware of specific advertisements.

However, as 404Media previously reported, Meta appears to have different standards for ad moderation. Policy-breaking content published by users as Facebook or Instagram posts will often be automatically detected by Meta and subsequently removed. Yet, that same content will not be detected when uploaded to Meta's advertising platform.

Over the past year, Meta has rolled out new tools and policies to prevent harmful online behavior such as sextortion. But Crush AI's ads are an example of how Meta's platforms are being weaponized to promote the very behavior that Meta says it wants to protect its young users from.