Algorithms revealed error rates of between 5%-50% when comparing photos of people wearing digitally created masks with unmasked faces.
Face masks not only have shown in research to slow the spread of the virus, they also limit facial-recognition technology from correctly identifying people, says a new study.
Research from the US National Institute of Standards & Technology (NIST) found that even the best of 89 commercial facial recognition algorithms tested had error rates between 5% & 50%, when matching people in digitally applied face masks with photos of the same person without a mask.
Masked
This study shows the limits of facial-recognition technology in a post-pandemic world, & are aimed at developing & testing technology that takes into consideration how to identify people who are masked, commented Mei Ngan, a NIST computer scientist & the report’s author, in a press statement.
She explained COVID-19 has given researchers developing facial-recognition technology a new factor to consider when creating accurate facial recognition algorithms.
“We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks,” she further mentioned. “Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind.”
Algorithms
NIST researchers used what is called “one-to-one” matching to test the algorithms, which means they compared a photo of someone with a different photo of that same person. They digitally applied masks using 9 mask variants in black or light blue to imitate the surgical-style masks most people wear now in public.
Results of the tests showed some challenges for developers of facial-recognition algorithms to consider.
Accuracy
The key overall finding is that algorithm accuracy when applied to masked faces declines considerably, researchers observed.
Even the most accurate algorithms failed to authenticate a person about 0.3% of the time, while top face-recognition technology had a failure rate of circa 5%, according to the study. Moreover, many algorithms considered competent by researchers failed at a much higher rate, between 20%-50%of the time.
Template
Masks also prevented the algorithms from even performing their task at all, registering what is called a “failure to enrol or template,” researchers suggested. This basically means the technology could not effectively measure a face’s features, such their size & distance from one another, for example, & then compare these measurements to those from another photo, which is how they work.
Also, masks that cover more of a person’s face made it harder for the algorithms to work, researchers discovered. The more a person’s nose was covered, the less the accuracy of the technology was.
Colour
The shape & colour of a mask also were factors when it came to accuracy. Round masks produced lower error rates, while black masks gave the algorithms more difficulty in recognizing faces than blue ones, researchers said.
False negative readings in which an algorithm fails to match 2 photos of the same person also increased when photo subjects wore masks, says the study. False positives, when an algorithm incorrectly indicates a match between photos of two different people, were less affected, however, with results generally the same as in unmasked situations, or slightly declined.
Accurate
The fact that facial recognition is less accurate when people are masked may be good for some, as the technology itself has been controversial & even shown evidence of wild inaccuracy in the best of cases.
Facial recognition has come under heavy fire from privacy advocates, particularly in its use by law enforcement, which has resulted in alleged racial profiling & errors that misidentified politicians as criminals. These scenarios, which involved Amazon’s Recognition platform even made some Democratic lawmakers propose a US nationwide ban on the technology.
Guidelines
Ngan also added that the results of the NIST study are not meant to be conclusive, but merely guidelines for developing technology specially designed to recognise faces whilst people are wearing masks
“We can draw a few broad conclusions from the results, but there are caveats,” she commented. “None of these algorithms were designed to handle face masks, & the masks we used are digital creations, not the real thing.”
