Why We Shouldn’t Trust Facial Recognition Systems of Police in Real-World Use?
Facial recognition technology is increasingly being presented as a powerful tool for modern policing. Police departments can use software to compare a face captured by a CCTV camera, mobile phone, body camera or other surveillance system against photographs stored in government or law-enforcement databases. In theory, the technology can help investigators identify unknown suspects, locate missing people and generate investigative leads much faster than conventional methods. But the central problem is that a facial-recognition “match” is not the same thing as proof of identity. When a probabilistic technology is treated as though it were certain evidence, a technical error can quickly become a police investigation, an arrest and potentially a criminal prosecution.
The distinction between technological performance in a controlled laboratory and performance on an actual street is particularly important. The U.S. Government Accountability Office has noted that while the accuracy of biometric identification technologies has improved in laboratory research, important gaps remain in understanding how they perform under real-world conditions. Real-world police images are rarely perfect passport-style photographs. They may be captured from a distance, at an angle, in poor lighting, through glass, partially obscured by masks or hair, or at a low resolution. Movement, shadows, camera quality and compression can further degrade the image. NIST’s current evaluation materials specifically acknowledge that poor image quality can increase false-negative errors and that demographic effects can remain relevant even when image quality is good.
The most serious danger is a false positive: the system identifies an innocent person as a possible match. In ordinary consumer technology, a false match may simply mean that a phone does not unlock correctly or that a system asks for another attempt. In policing, the consequences can be dramatically different. NIST has emphasized that in a one-to-many identification search, a false positive can place an innocent person on a list of candidates for further investigation. Once that person’s name enters a police investigation, however, the algorithmic suggestion can acquire a psychological authority that it does not deserve.
This creates what researchers and investigators have described as a form of automation bias: people can place excessive confidence in a computer-generated result, even when other evidence contradicts it. The danger is not necessarily that police officers deliberately abandon professional judgment. Rather, an apparently scientific result can influence how subsequent evidence is interpreted. Once an individual has been identified as a “match,” investigators may unconsciously begin looking for evidence that supports the identification rather than asking whether the person actually committed the crime.
The real-world consequences are no longer theoretical. Robert Williams, a Detroit resident, was wrongfully arrested in 2020 after police used facial-recognition technology to connect him to surveillance footage from a watch theft. The image was poor, and the algorithm’s identification was wrong. Williams was detained despite being innocent, and the incident became one of the best-known examples of the danger of relying on facial recognition in a criminal investigation. A subsequent settlement led to stricter requirements for Detroit police, including greater disclosure of the limitations of facial-recognition searches and additional safeguards around their use.
The problem did not end with Williams. In 2023, Detroit police arrested Porcha Woodruff, who was eight months pregnant, after facial-recognition technology was used in an investigation into a carjacking. Police later acknowledged that she was the wrong suspect and prosecutors dropped the charges. A federal judge dismissed her civil-rights lawsuit in 2025, finding that the legal standard for liability had not been established, but the case nevertheless illustrates the enormous human consequences that can follow when an algorithmic lead becomes intertwined with conventional police procedures. Detroit subsequently changed its policy so that officers cannot make arrests solely on the basis of facial-recognition results or photo lineups generated from such searches.
There is also a significant demographic dimension to the problem. NIST’s extensive testing has found demographic differences in the performance of many facial-recognition algorithms. Its research found that false-positive rates can vary substantially across demographic groups, although the size and direction of the difference depend on the particular algorithm and application. NIST has also found that image quality, age, sex and race can affect error rates. Its current evaluation data continue to document demographic differentials in both false-positive and false-negative performance.
This does not mean that every facial-recognition system is equally inaccurate or that every demographic group is misidentified by every algorithm. That would be an oversimplification. The technology has improved considerably, and some algorithms perform much better than others. NIST’s research itself demonstrates substantial variation between algorithms. The important point is that “facial recognition” is not a single technology with one universal accuracy rate. Performance depends on the algorithm, database, image quality, demographic composition, threshold selected and the precise task being performed.
That distinction becomes crucial when police departments publicize impressive-looking accuracy percentages. An algorithm might perform extremely well when comparing two high-quality photographs under controlled conditions but perform differently when asked to identify one person from thousands or millions of images using a blurry surveillance photograph. The statistical environment changes as the size of the search database changes. Even a very small false-positive rate can produce potentially significant numbers of incorrect candidates when a system searches enormous databases.
Another problem is that police officers and the public may not always know exactly what a particular facial-recognition result means. A similarity score is not necessarily a probability that the suspect is the person shown in the image. A high similarity score does not automatically establish that two photographs depict the same person, just as a low score does not necessarily establish that they depict different people. The interpretation depends on the algorithm, threshold, database and circumstances of the search. Treating a numerical score as if it were an objective statement of identity can therefore be dangerously misleading.
Transparency is another major concern. A person accused of a crime may not be able to meaningfully challenge an algorithmic identification if investigators do not disclose which system was used, what image was submitted, what threshold was applied, what alternatives were returned and how the result was interpreted. The U.S. Government Accountability Office has repeatedly highlighted the need for law-enforcement agencies to understand and manage the accuracy, privacy and civil-rights risks associated with facial-recognition systems. In 2024, GAO reported that seven federal law-enforcement agencies it reviewed had initially used facial-recognition services without requiring relevant staff training, while some agencies lacked policies specifically addressing facial recognition and civil rights.
The privacy issue is equally profound. A conventional eyewitness may identify one person connected with one particular investigation. A facial-recognition network can potentially search huge numbers of faces captured from public or private cameras. This changes the nature of surveillance itself. Instead of investigating a known suspect after a crime has occurred, authorities can acquire the technical ability to search for people across large collections of images. The Government Accountability Office has identified concerns including the inability of people to remain anonymous in public and the collection or storage of facial images.
This is particularly sensitive when facial recognition is deployed at protests, political gatherings, religious events or other lawful public assemblies. The technology can transform ordinary participation in public life into a searchable biometric record. GAO reported that several federal agencies had used facial-recognition technology in connection with civil unrest and protests following the death of George Floyd in 2020. Such applications demonstrate why the question is not merely whether the software can identify faces, but whether authorities should be permitted to identify people in particular circumstances in the first place.
India is now confronting many of these questions as police forces increasingly use facial-recognition systems. Recent reporting surrounding protests at Delhi’s Jantar Mantar has again raised concerns about the reliability and appropriate use of facial recognition in crowded public spaces. The Indian Express reported that police used facial-recognition technology to identify people during the protests and highlighted concerns involving false positives, image quality, viewing angles, lighting and the absence of a universally standardized threshold for determining when a facial-recognition result should be treated as a positive identification.
The Delhi episode is particularly instructive because crowded demonstrations represent almost the opposite of the conditions under which facial-recognition systems are easiest to evaluate. People move continuously, cameras capture faces at different angles, individuals may partially obscure their faces and surveillance footage can be low quality. A system may generate a candidate list, but the technological process cannot by itself establish that a particular individual committed a particular offence. Even Delhi Police’s reported position, according to recent coverage, is that facial recognition should function as an investigative aid rather than the sole basis for action, with field-level verification required.
That principle should be fundamental everywhere: a facial-recognition result should be treated as an investigative lead, not as conclusive evidence of guilt. Independent evidence should be required before an individual is arrested, searched, charged or subjected to other coercive action. Such evidence might include reliable eyewitness testimony, physical evidence, verified location information, digital evidence, fingerprints, DNA or other information that independently connects the individual to the alleged offence. The more serious the consequence, the stronger the requirement for independent corroboration should be.
There is also a danger in assuming that adding a human officer automatically solves the problem. Human review is necessary, but humans can also be influenced by machine-generated recommendations. If an officer is shown a computer-selected suspect and then asked to confirm the identification, the process can become less independent than it appears. A genuinely meaningful safeguard therefore requires investigators to evaluate contradictory evidence rather than merely confirming the computer’s suggestion.
The proper question, therefore, is not whether facial recognition is “good” or “bad” technology. It is whether the technology is being used for a task in which its particular error characteristics are acceptable and whether sufficient safeguards exist to prevent an error from becoming a deprivation of liberty. GAO has similarly emphasized that forensic algorithms can provide benefits to investigations while also presenting challenges involving bias, misuse, transparency and communication of results.
Facial recognition can undoubtedly assist law enforcement. It can help investigators narrow an enormous pool of photographs, identify possible persons of interest and accelerate investigations that would otherwise take considerable time. Rejecting every technological tool simply because it can make mistakes would not be sensible. Fingerprints, DNA analysis, eyewitness testimony and other forms of evidence can also be wrong or misinterpreted. The crucial difference is that society has developed legal and procedural safeguards around many traditional forms of evidence. Facial recognition needs comparable scrutiny rather than being granted special authority because its output comes from a computer.
The strongest safeguard is therefore simple but demanding: no person should lose their liberty merely because a facial-recognition system says they look like someone else. The technology can point investigators toward a possibility, but it should not decide who is guilty. Before police act against an individual, investigators must establish independent evidence, document how the facial-recognition search was conducted, preserve the original image and search results, disclose relevant limitations and allow the identification to be challenged.
The greatest risk of facial recognition is not that machines will become perfect at identifying people. It is that society will become too willing to treat imperfect machine predictions as facts. A computer-generated match may look objective because it is expressed through numbers, scores and sophisticated software, but an algorithm can only produce a result from its training data, image quality, database and programmed methodology. When that result enters the criminal-justice system, the cost of being wrong is measured not merely in technical error rates but in people’s freedom, reputation, privacy and dignity. For that reason, facial recognition should remain a carefully controlled investigative tool—not an unquestioned authority on human identity.
