Apple's new feature scans for child abuse images

Apple is officially taking on child predators with new safety features for iPhone and iPad.

One scans for child sexual abuse material (CSAM), which sounds like a good thing. But it has several privacy experts concerned.

So, how does it work? The feature, available on iOS 15 and iPadOS 15 later this year, uses a new proprietary technology called NeuralHash to detect known CSAM images.

Before the image is stored in iCloud Photos, it goes through a matching process on the device against specific CSAM hashes.

It then uses technology called "threshold secret sharing," which doesn't allow Apple to interpret a photo unless the related account has crossed a threshold of CSAM content.

Apple can then report any CSAM content it finds to the National Center for Missing and Exploited Children (NCMEC).

It's worth noting that there is room for false positives. Matthew Green, cybersecurity expert and associate professor at Johns Hopkins University, took to Twitter to voice his concerns.

This Tweet is currently unavailable. It might be loading or has been removed.

“To say that we are disappointed by Apple’s plans is an understatement,” said the Electronic Frontier Foundation, arguing that “even a thoroughly documented, carefully thought-out, and narrowly-scoped backdoor is still a backdoor.”

We've reached out to Apple for comment and will update this story when we hear back.

Apple says its threshold provides "an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account."

Once a device crosses that threshold, the report is manually reviewed. If Apple finds a match, it disables the user's account and a report is sent to NCMEC. Users who think their account has been flagged by mistake will have to file an appeal in order to get it back.

While it's tough to criticize a company for wanting to crack down on child pornography, the fact that Apple has the ability to scan someone's photos in general is concerning. It's even worse to think that an actual human being might look through private images only to realize an account was mistakenly identified.

It's also ironic that Apple, the company that brags about its privacy initiatives, specifically its Nutrition Labels and App Transparency Tracking, has taken this step.

Apple assures users that "CSAM is designed with user privacy in mind," which is why it matches an image on-device before it's sent to iCloud Photos. But they said the same thing about AirTags, and, well, those turned out to be a privacy nightmare.