Facial recognition could soon be used to identify masked protesters
"V for Vendetta" masks are a typical feature of many political protests since the eponymous dystopian movie came out in 2005 -- but what if facial recognition technology was able to identify the face behind the mask?
We're not there yet, but researchers are slowly and steadily making highly-controversial steps in this direction.
Academics from Cambridge University, India's National Institute of Technology, and the Indian Institute of Science used deep learning and a dataset of pictures of people in disguise to try to identify masked faces with an acceptable level of reliability.
The research, published on the preprint server arXiv and shared in an AI newsletter, went viral after prominent academic and sociologist Zeynep Tufekci shared it on Twitter.
This Tweet is currently unavailable. It might be loading or has been removed.
Stressing that the paper "isn't that great", Tufekci nonetheless points out that it's the direction that's worrying, as oppressive and authoritarian states could use the tool to stifle dissent and expose anonymous protesters.
In the study, researchers introduced a deep learning technique to detect 14 key areas of the face -- mainly around the eyes, the tip of the nose and corners of the mouth -- and then trained an algorithm to identify a face by forming a "star-net structure".
The neural networks estimate the facial keypoints even when the face is disguised by caps, scarves and glasses.
In this series of picture, for instance, the algorithm measures the distance between facial keypoints and then uses it to identify them again in other images:
When we look at the results, though, they're lagging behind industry standards by a significant proportion.
In case of a face concealed by a hat or scarf against photos of five people, the model was able to identify the person 56% of the time. With the combination of hat, scarf and glasses, the percentage fell to just 43%.
Experts like Patrick Huber, researcher at the University of Surrey, criticised the paper on The Verge saying it's not "particularly convincing" and pointing out that using keypoints is not exactly the same as matching faces to mugshots.
Amarjot Singh, lead researcher of the study, agrees that the system is not at the point where it can be used in practice, but urges people not to dismiss the research.
"This, in my opinion, is the first promising AI (deep learning) based work that solved the problem of DFI (Disguised Face Identification), with a reasonable accuracy," he tells Mashable. "It will get the ball rolling towards solving the DFI task."
"The accuracy is not high enough as normal Face Recognition system but this isn't the main problem," he continued. "Our AI model is able to see and predict where a specific face part could have been even if it is not visible making it way superior to other Face Recognition systems."
"The accuracy is less because it is a very difficult problem!"
A crackdown on dissent?
Facial recognition has been in use in UK and US enforcement agencies for a while.
In Britain, the first arrest attributed to it is from June this year, though in 2015 Leicestershire police raised a controversy by scanning the faces of 90,000 festival-goers at Download Festival, checking them against a list of wanted criminals across the country.
According to a study published by researchers at Georgetown University's Center on Privacy and Technology, about half of American adults — 117 million Americans — are already in a "law enforcement face recognition network."
Amarjot Singh agrees with Tufkeci's concerns that the system may be "misused" if they fall into the hands of authoritarian regimes, but points out that the models "can be used to take many criminals off the streets as well."
But is the system able to identify protesters wearing Anonymous' Guy Fawkes style masks?
Not yet, but we're getting there, the researcher said.
"The system currently doesn't work for these rigid masks as the algorithm tries to capture the structure of face of the person using facial keypoints," he says.
"The structure of the face is not destroyed if only a part of the face is covered and the AI model can still predict where the hidden face part is supposed to be. However, the Guy Fawkes mask will cover the whole face which will make it difficult for the AI model to predict these keypoints. It is equivalent to someone getting a plastic surgery which changes the structure of the face."
"We are currently working on accommodating the capability to work with rigid masks as well. We hope the new model will get this capability with the next few months," he concluded.
