Defending Expertise in Fingerprint Identification
In 2011, we published the results of an experiment in Psychological Science where we compared the fingerprint matching performance of experts and novices. The article can be accessed here. Psychological Science is a high impact journal that requires very short articles. As a result, we had very little room to present our expertise results and also put them into context.
On this web page, we first describe the experiment and the take-home messages. We then provide a working draft of a manuscript where we describe our experiment, the results, and the implications in a form that will be particularly relevant to fingerprint examiners, forensic managers, and the legal community. The content on this page will be continuously updated until its eventual publication in an academic journal; the working title is “Defending Expertise in Fingerprint Identification.”
The Experiment
In our experiment, we were measuring the benefit of expertise in matching fingerprints rather than accuracy per se. The difficulty is that no properly controlled experiments have been conducted on fingerprint examiners’ accuracy in identifying perpetrators, even though fingerprints have been used in criminal courts for more than 100 years. We tested fingerprint examiners at police stations across Australia: in Queensland, New South Wales, Victoria, South Australia, and the Australian Federal Police in Canberra. We put examiners in a situation similar to their usual work, but we maintained tight experimental control by using simulated crime-scene prints and highly similar AFIS distractors in a signal detection paradigm.
The Results
We show that qualified, court-practicing fingerprint experts are exceedingly accurate compared with novices, but are not infallible. Our experts tended to err on the side of caution by making errors that would free the guilty rather than convict the innocent. They occasionally made the kind of error that can lead to false convictions.
Error Rates
Inferring, from our results, that “fingerprint examiners are 99.32% accurate” would be disingenuous. Any claim of accuracy, based on our results, would have to be followed by a list of qualifiers. For example, “Some qualified fingerprint examiners are 99.32% accurate at correctly declaring nonmatching prints as such when the prints were obtained from the most similar non-match according to the NAFIS system, and when examiners are not provided with their usual tools, or independent verification,” and so on. The qualifiers are limitations only if you’re trying to make an overgeneralization like, “fingerprint examiners are 99.32% accurate,” which is close to impossible to make in any area of expertise (let alone on the basis of our results).
We can, however, legitimately conclude that experts are more accurate (and conservative) than novices, for example. If this is what we’re concluding (and we are), then all of the qualifying remarks above are completely irrelevant. We have demonstrated that experts can make errors in certain situations, but that doesn’t mean that experts do make errors in practice.
Expertise
Our use of naive undergraduates as a comparison group was deliberate. A comparison to novices shows that our task was difficult enough for experts to perform accurately but for novices to perform relatively poorly. This distinction is fundamental to the question of expert testimony because it demonstrates specialized knowledge, which could be called upon to satisfy legal admissibility criteria. Indeed, we conclude that “Qualified fingerprint examiners now have evidence to legitimately claim specialized knowledge, which may satisfy legal admissibility criteria.”
A Research Culture
Considering the central role of humans in forensic identification, the field would benefit from further psychological research. From here we can ask more sophisticated questions than just those about error rates and expertise. For example, what’s the most effective way to train novices? What information is the most important for matching or excluding prints? What elements of the matching task best distinguish experts and novices? How does expertise with fingerprints develop over time? What’s the relationship between the Analysis and Comparison phase of he identification process? What’s best practice in providing feedback and self-assessment? What’s the most effective way to present fingerprint evidence to juries?
The practical outcomes from answering questions such as these include a better understanding of the source of identification errors and factors that influence performance, more effective recruitment and training methods, and greater validity in presenting forensic evidence in court.
Research on clinical reasoning in medicine, for example, developed over the past 40 years, after it became evident that physicians’ decisions too often resulted in adverse consequences for patients. Much has been learned about differences between novice and expert medical practitioners, the influence of cognitive biases in medical decision making, and the most effective ways to incorporate such knowledge into practice. Further research into forensic decision making will help to ensure the integrity of forensics as an investigative tool.
Responding to the NAS
Researchers at the National Academy of Sciences (NAS) and elsewhere have argued that there is an urgent need to develop objective measures of accuracy in fingerprint identification and have called for experiments on expertise in forensic examination. We have several planned research directions to directly address the mandate by the NAS for “more and better research” in forensic science. This work will allow police, intelligence systems, and investigators to interpret evidence more effectively and efficiently, assist forensic examiners in the development of evidence-based training programs, discourage exaggerated interpretations of forensic evidence, and aid in the development of a model of expert testimony that does not extend beyond the capabilities of examiners or beyond the scope of our experimental findings. We are also building a free and open-source database with standardized sets of biometric material—The Forensics Informatics Biometric Repository (FIB-R)—and launching The Forensic Reasoning Project.
We hope this approach will lead to a culture of cooperation between impeding researchers and forensic experts, rather than promote the adversarial system currentlyadvancement of the field.
Expertise in Fingerprint Identification
A manuscript detailing the issues above, and many others, has been submitted for publication. It is titled, “Expertise in Fingerprint Identification,” by Thompson, Tangen and McCarthy. The paper will appear on this website once published.