The 1975 codification of the law of evidence, in the Federal Rules of Evidence, introduced a subtle, aspirational criterion for expert witness opinion – knowledge. As originally enacted, Rule 702 read:

“If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise.”[1]

In case anyone missed the point, the Advisory Committee Note for the original Rule 702 emphasized that the standard was an epistemic standard:

“An intelligent evaluation of facts is often difficult or impossible without the application of some scientific, technical, or other specialized knowledge. The most common source of this knowledge is the expert witness, although there are other techniques for supplying it.”[2]

Perhaps we should not be too surprised that the epistemic standard was missed by most judges, and even by most lawyers. For a very long time, the common law set out a minimal test for expert witness opinion testimony. The expert witness had to be qualified by training, experience, or education, and the opinion proffered had to be logically and legally relevant to the issues in the case.[3] The enactment of Rule 702, in 1975, barely made a dent in the regime of easy admissibility.

Before the Federal Rules of Evidence, there was, of course, the famous Frye case, which involved an appeal from the excluded expert witness opinion based upon William Marston’s polygraph machine. In 1923, the court in Frye affirmed the exclusion of the expert witness opinion, based upon the lack of general acceptance of the device’s reliability, with its famous twilight zone language:[4]

“Just when a scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone the evidential force of the principle must be recognized, and while courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.”

With the explosion of tort litigation fueled by strict products liability doctrine, lawyers pressed Frye’s requirement of general acceptance into service as a bulwark against unreliable scientific opinions. Many courts, however, limited Frye to novel devices, and in 1993, the Supreme Court, in Daubert,[5] rejected the legal claim that Rule 702 had incorporated the common law “general acceptance” test. Looking to the language of the rule itself, the Supreme Court discerned that the rule laid down an epistemic test, not a call for sociological surveys about the prevalence of beliefs.

Resistance to the spirit and text of Rule 702 has been widespread and deep seated. After Daubert, the Supreme Court decided three more cases to emphasize that the epistemic standard was “exacting” and that it would not go away.[6] Since Daubert was decided in 1993, Rule 702 was amended substantively, in 2000, to incorporate some of the essence of the Supreme Court’s quartet,[7] which required the proponent of expert witness opinion to establish that proffered testimony is based upon sufficient facts or data, is the product of reliable principles and methods, and the result of reliably applying those reliable principles and methods to the facts of the case.

The change in the law of expert witnesses, in the 1990s, left some academic commentators well-nigh apoplectic. One professor of evidence law at a large law school complained that the law was a “conceptual muddle containing within it a threat to liberty and popular participation in government.”[8] Many federal district and intermediate appellate courts responded by ignoring the language of Rule 702, by reverting to pre-Daubert precedent, or by inventing new standards and shifting the burden to the party challenging the expert witness opinion’s admissibility. For many commentators, lawyers, and judges, science had no validity concerns that the law was bound to respect.

The judicial evasion and avoidance of the requirements of Rule 702 did not go unnoticed. Professor David Bernstein and practicing lawyer Eric Lasker wrote a paper in 2015, to call attention to the judicial disregard of the requirements of Rule 702.[9]  Several years of discussion and debate ensued before the Judicial Conference Advisory Committee on Evidence Rules (AdCom), in 2021, acknowledged that “in a fair number of cases, the courts have found expert testimony admissible even though the proponent has not satisfied the Rule 702(b) and (d) requirements by a preponderance of the evidence.”[10] This frank acknowledgement led the AdCom to propose amending Rule 702, “to clarify and emphasize” that gatekeeping requires determining whether the proponent has demonstrated to the court “that it is more likely than not that the proffered testimony meets the admissibility requirements set forth in the rule.”[11]  The Proposed Committee Note written in support of amending Rule 702 observed that “many courts have held that the critical questions of the sufficiency of an expert’s basis, and the application of the expert’s methodology, are questions of weight and not admissibility. These rulings are an incorrect application of Rules 702 and 104(a).”[12]

The proposed new Rule 702 is now law,[13] with its remedial clarification that the proponent of expert witness opinion must show the court that the opinion is sufficiently supported by facts or data,[14] that the opinion is “the product of reliable principles and methods,”[15]  and that the opinion “reflects a reliable application of the principles and methods to the facts of the case.”[16] The Rule prohibits deferring the evaluation of sufficiency of support or reliability of application of method to the trier of fact; there is no statutory support for suggesting that these inquires always or usually go to “weight and not admissibility,” or that there is a presumption of admissibility.

We may not have reached the Age of Aquarius, but the days of “easy admissibility” should be confined to the dustbin of legal history. Rule 702 is quickly approaching its 50th birthday, with the last 30 years witnessing the implementation of the promise and potential of an epistemic standard of trustworthiness for expert witness opinion testimony. Rule 702, in its present form, should go a long way towards putting validity questions squarely before the court under Rule 702. Nullius in verba[17] has been the motto of the Royal Society since 1660; it should now guide expert witness practice in federal court going forward.

[1] Pub. L. 93–595, §1, Jan. 2, 1975, 88 Stat. 1937 (emphasis added).

[2] Notes of Advisory Committee on Proposed Rules (1975) (emphasis added).

[3] See Charles T. McCormick, Handbook of the Law of Evidence 28-29, 363 (1954) (“Any relevant conclusions which are supported by a qualified expert witness should be received unless there are other reasons for exclusion.”)

[4] Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923).

[5] Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993).

[6] General Electric Co. v. Joiner, 522 U.S. 136 (1997); Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999); Weisgram v. Marley Co., 528 U.S. 440 (2000).

[7] See notes 5, 6, supra.

[8] John H. Mansfield, “An Embarrassing Episode in the History of the Law of Evidence,” 34 Seton Hall L. Rev. 77, 77 (2003); see also John H. Mansfield, “Scientific Evidence Under Daubert,” 28 St. Mary’s L.J. 1, 23 (1996). Professor Mansfield was the John H. Watson, Jr., Professor of Law, at the Harvard Law School. Many epithets were thrown in the heat of battle to establish meaningful controls over expert witness testimony. See, e.g., Kenneth Chesebro, “Galileo’s Retort: Peter Huber’s Junk Scholarship,” 42 Am. Univ. L. Rev. 1637 (1993). Mr. Chesebro was counsel of record for plaintiffs-appellants in Daubert, well before he became a convicted racketeer in Georgia.

[9] David Bernstein & Eric Lasker, “Defending Daubert: It’s Time to Amend Federal Rules of Evidence 702,” 57 Wm. & Mary L Rev. 1 (2015).

[10] Report of AdCom (May 15, 2021), at https://www.uscourts.gov/rules-policies/archives/committee-reports/advisory-committee-evidence-rules-may-2021. See also AdCom, Minutes of Meeting at 4 (Nov. 13, 2020) (“[F]ederal cases . . . revealed a pervasive problem with courts discussing expert admissibility requirements as matters of weight.”)], at https://www.uscourts.gov/rules-policies/archives/meeting-minutes/advisory-committee-evidence-rules-november-2020.

[11] Proposed Committee Note, Summary of Proposed New and Amended Federal Rules of Procedure (Oct. 19, 2022), at https://www.uscourts.gov/sites/default/files/2022_scotus_package_0.pdf

[12] Id. (emphasis added).

[13] In April 2023, Chief Justice Roberts transmitted the proposed Rule 702, to Congress, under the Rules Enabling Act, and highlighted that the amendment “shall take effect on December 1, 2023, and shall govern in all proceedings thereafter commenced and, insofar as just and practicable all proceedings then pending.” S. Ct. Order, at 3 (Apr. 24, 2023), https://www.supremecourt.gov/orders/courtorders/frev23_5468.pdf; S.Ct. Transmittal Package (Apr. 24, 2023), < https://www.uscourts.gov/sites/default/files/2022_scotus_package_0.pdf>.

[14] Rule 702(b).

[15] Rule 702(c).

[16] Rule 702(d).

[17] Take no one’s word for it.

The current issue of the Proceedings of the National Academies of Science (PNAS) features a medley of articles on science generally, and forensic science, in the law.[1] The general editor of the compilation appears to be editorial board member, Thomas D. Albright, the Conrad T. Prebys Professor of Vision Research at the Salk Institute for Biological Studies.

 I have not had time to plow through the set of offerings, but even a superficial inspection reveals that the articles will be of interest to lawyers and judges involved in the litigation of scientific issues. The authors seem to agree that descriptively and prescriptively, validity is more important than expertise in the legal  consideration of scientific evidence.

1. Thomas D. Albright, “A scientist’s take on scientific evidence in the courtroom,” 120 Proceedings of the National Academies of Science 120 (41) e2301839120 (2023).

Albright’s essay was edited by Henry Roediger, a psychologist at the Washington University in St. Louis.

Abstract

Scientific evidence is frequently offered to answer questions of fact in a court of law. DNA genotyping may link a suspect to a homicide. Receptor binding assays and behavioral toxicology may testify to the teratogenic effects of bug repellant. As for any use of science to inform fateful decisions, the immediate question raised is one of credibility: Is the evidence a product of valid methods? Are results accurate and reproducible? While the rigorous criteria of modern science seem a natural model for this evaluation, there are features unique to the courtroom that make the decision process scarcely recognizable by normal standards of scientific investigation. First, much science lies beyond the ken of those who must decide; outside “experts” must be called upon to advise. Second, questions of fact demand immediate resolution; decisions must be based on the science of the day. Third, in contrast to the generative adversarial process of scientific investigation, which yields successive approximations to the truth, the truth-seeking strategy of American courts is terminally adversarial, which risks fracturing knowledge along lines of discord. Wary of threats to credibility, courts have adopted formal rules for determining whether scientific testimony is trustworthy. Here, I consider the effectiveness of these rules and explore tension between the scientists’ ideal that momentous decisions should be based upon the highest standards of evidence and the practical reality that those standards are difficult to meet. Justice lies in carefully crafted compromise that benefits from robust bonds between science and law.

2. Thomas D.Albright, David Baltimore, Anne-MarieMazza, Jennifer Mnookin, and David S. Tatel, “Science, evidence, law, and justice,” 120 Proceedings of the National Academies of Science 120 (41) e2301839120 (2023).

Professor Baltimore is a nobel laureate and researcher in biology, now at the California Institute of Technology. Anne-Marie Mazza is the director of the Committee on Science, Technology, and Law, of the National Academies of Sciences, Engineering, and Medicine. Jennifer Mnookin is the chancellor of the University of Wisconsin, Madison; previously, she was the dean of the UCLA School of Law. Judge Tatel is a federal judge on the United States Court of Appeals for the District of Columbia Circuit.

Abstract

For nearly 25 y, the Committee on Science, Technology, and Law (CSTL), of the National Academies of Sciences, Engineering, and Medicine, has brought together distinguished members of the science and law communities to stimulate discussions that would lead to a better understanding of the role of science in legal decisions and government policies and to a better understanding of the legal and regulatory frameworks that govern the conduct of science. Under the leadership of recent CSTL co-chairs David Baltimore and David Tatel, and CSTL director Anne-Marie Mazza, the committee has overseen many interdisciplinary discussions and workshops, such as the international summits on human genome editing and the science of implicit bias, and has delivered advisory consensus reports focusing on topics of broad societal importance, such as dual use research in the life sciences, voting systems, and advances in neural science research using organoids and chimeras. One of the most influential CSTL activities concerns the use of forensic evidence by law enforcement and the courts, with emphasis on the scientific validity of forensic methods and the role of forensic testimony in bringing about justice. As coeditors of this Special Feature, CSTL alumni Tom Albright and Jennifer Mnookin have recruited articles at the intersection of science and law that reveal an emerging scientific revolution of forensic practice, which we hope will engage a broad community of scientists, legal scholars, and members of the public with interest in science-based legal policy and justice reform.

3. Nicholas Scurich, David L. Faigman, and Thomas D. Albright, “Scientific guidelines for evaluating the validity of forensic feature-comparison methods,” 120 Proceedings of the National Academies of Science (2023).

Nicholas Scurich is the chair of the department of Psychological Science, at the University of Southern California, David Faigman has written prolifically about science in the law. He is now the chancellor and dean, at the University of San Francisco College of Law.

Abstract

When it comes to questions of fact in a legal context—particularly questions about measurement, association, and causality—courts should employ ordinary standards of applied science. Applied sciences generally develop along a path that proceeds from a basic scientific discovery about some natural process to the formation of a theory of how the process works and what causes it to fail, to the development of an invention intended to assess, repair, or improve the process, to the specification of predictions of the instrument’s actions and, finally, empirical validation to determine that the instrument achieves the intended effect. These elements are salient and deeply embedded in the cultures of the applied sciences of medicine and engineering, both of which primarily grew from basic sciences. However, the inventions that underlie most forensic science disciplines have few roots in basic science, and they do not have sound theories to justify their predicted actions or results of empirical tests to prove that they work as advertised. Inspired by the “Bradford Hill Guidelines”—the dominant framework for causal inference in epidemiology—we set forth four guidelines that can be used to establish the validity of forensic comparison methods generally. This framework is not intended as a checklist establishing a threshold of minimum validity, as no magic formula determines when particular disciplines or hypotheses have passed a necessary threshold. We illustrate how these guidelines can be applied by considering the discipline of firearm and tool mark examination.

4. Peter Stout, “The secret life of crime labs,” 120 Proceedings of the National Academies of Science 120 (41) e2303592120 (2023).

Peter Stout is a scientist with the Houston Forensic Science Center, in Houston, Texas. The Center describes itself as “an independent local government corporation,” which provides forensic “services” to the Houston police

Abstract

Houston TX experienced a widely known failure of its police forensic laboratory. This gave rise to the Houston Forensic Science Center (HFSC) as a separate entity to provide forensic services to the City of Houston. HFSC is a very large forensic laboratory and has made significant progress at remediating the past failures and improving public trust in forensic testing. HFSC has a large and robust blind testing program, which has provided many insights into the challenges forensic laboratories face. HFSC’s journey from a notoriously failed lab to a model also gives perspective to the resource challenges faced by all labs in the country. Challenges for labs include the pervasive reality of poor-quality evidence. Also that forensic laboratories are necessarily part of a much wider system of interdependent functions in criminal justice making blind testing something in which all parts have a role. This interconnectedness also highlights the need for an array of oversight and regulatory frameworks to function properly. The major essential databases in forensics need to be a part of blind testing programs and work is needed to ensure that the results from these databases are indeed producing correct results and those results are being correctly used. Last, laboratory reports of “inconclusive” results are a significant challenge for laboratories and the system to better understand when these results are appropriate, necessary and most importantly correctly used by the rest of the system.

5. Brandon L. Garrett & Cynthia Rudin, “Interpretable algorithmic forensics,” 120 Proceedings of the National Academies of Science 120 (41) 120 (41) e2301842120 (2023).

Garrett teaches at the Duke University School of Law. Rudin teaches statistics at Duke University.

Abstract

One of the most troubling trends in criminal investigations is the growing use of “black box” technology, in which law enforcement rely on artificial intelligence (AI) models or algorithms that are either too complex for people to understand or they simply conceal how it functions. In criminal cases, black box systems have proliferated in forensic areas such as DNA mixture interpretation, facial recognition, and recidivism risk assessments. The champions and critics of AI argue, mistakenly, that we face a catch 22: While black box AI is not understandable by people, they assume that it produces more accurate forensic evidence. In this Article, we question this assertion, which has so powerfully affected judges, policymakers, and academics. We describe a mature body of computer science research showing how “glass box” AI—designed to be interpretable—can be more accurate than black box alternatives. Indeed, black box AI performs predictably worse in settings like the criminal system. Debunking the black box performance myth has implications for forensic evidence, constitutional criminal procedure rights, and legislative policy. Absent some compelling—or even credible—government interest in keeping AI as a black box, and given the constitutional rights and public safety interests at stake, we argue that a substantial burden rests on the government to justify black box AI in criminal cases. We conclude by calling for judicial rulings and legislation to safeguard a right to interpretable forensic AI.

6. Jed S. Rakoff & Goodwin Liu, “Forensic science: A judicial perspective,” 120 Proceedings of the National Academies of Science e2301838120 (2023).

Judge Rakoff has written previously on forensic evidence. He is a federal district court judge in the Southern District of New York. Goodwin Liu is a justice on the California Supreme Court. Their article was edited by Professor Mnookin.

Abstract

This article describes three major developments in forensic evidence and the use of such evidence in the courts. The first development is the advent of DNA profiling, a scientific technique for identifying and distinguishing among individuals to a high degree of probability. While DNA evidence has been used to prove guilt, it has also demonstrated that many individuals have been wrongly convicted on the basis of other forensic evidence that turned out to be unreliable. The second development is the US Supreme Court precedent requiring judges to carefully scrutinize the reliability of scientific evidence in determining whether it may be admitted in a jury trial. The third development is the publication of a formidable National Academy of Sciences report questioning the scientific validity of a wide range of forensic techniques. The article explains that, although one might expect these developments to have had a major impact on the decisions of trial judges whether to admit forensic science into evidence, in fact, the response of judges has been, and continues to be, decidedly mixed.

7. Jonathan J. Koehler, Jennifer L. Mnookin, and Michael J. Saks, “The scientific reinvention of forensic science,” 120 Proceedings of the National Academies of Science e2301840120 (2023).

Koehler is a professor of law at the Northwestern Pritzker School of Law. Saks is a professor of psychology at Arizona State University, and Regents Professor of Law, at the Sandra Day O’Connor College of Law.

Abstract

Forensic science is undergoing an evolution in which a long-standing “trust the examiner” focus is being replaced by a “trust the scientific method” focus. This shift, which is in progress and still partial, is critical to ensure that the legal system uses forensic information in an accurate and valid way. In this Perspective, we discuss the ways in which the move to a more empirically grounded scientific culture for the forensic sciences impacts testing, error rate analyses, procedural safeguards, and the reporting of forensic results. However, we caution that the ultimate success of this scientific reinvention likely depends on whether the courts begin to engage with forensic science claims in a more rigorous way.

8. William C. Thompson, “Shifting decision thresholds can undermine the probative value and legal utility of forensic pattern-matching evidence,” 120 Proceedings of the National Academies of Science e2301844120 (2023).

Thompson is professor emeritus in the Department of Criminology, Law & Society, University of California, Irvine.

Abstract

Forensic pattern analysis requires examiners to compare the patterns of items such as fingerprints or tool marks to assess whether they have a common source. This article uses signal detection theory to model examiners’ reported conclusions (e.g., identification, inconclusive, or exclusion), focusing on the connection between the examiner’s decision threshold and the probative value of the forensic evidence. It uses a Bayesian network model to explore how shifts in decision thresholds may affect rates and ratios of true and false convictions in a hypothetical legal system. It demonstrates that small shifts in decision thresholds, which may arise from contextual bias, can dramatically affect the value of forensic pattern-matching evidence and its utility in the legal system.

9. Marlene Meyer, Melissa F. Colloff, Tia C. Bennett, Edward Hirata, Amelia Kohl, Laura M. Stevens, Harriet M. J. Smith, Tobias Staudigl & Heather D. Flowe, “Enabling witnesses to actively explore faces and reinstate study-test pose during a lineup increases discriminability,” 120 Proceedings of the National Academies of Science e2301845120 (2023).

Marlene Meyer, Melissa F. Colloff, Tia C. Bennett, Edward Hirata, Amelia Kohl, and Heather D. Flowe are psychologists at the School of Psychology, University of Birmingham (United Kingdom). Harriet M. J. Smith is a psychologist in the School of Psychology, Nottingham Trent University, Nottingham, United Kingdom, and Tobias Staudigl is a psychologist in the Department of Psychology, Ludwig-Maximilians-Universität München, in Munich, Germany.

Abstract

Accurate witness identification is a cornerstone of police inquiries and national security investigations. However, witnesses can make errors. We experimentally tested whether an interactive lineup, a recently introduced procedure that enables witnesses to dynamically view and explore faces from different angles, improves the rate at which witnesses identify guilty over innocent suspects compared to procedures traditionally used by law enforcement. Participants encoded 12 target faces, either from the front or in profile view, and then attempted to identify the targets from 12 lineups, half of which were target present and the other half target absent. Participants were randomly assigned to a lineup condition: simultaneous interactive, simultaneous photo, or sequential video. In the front-encoding and profile-encoding conditions, Receiver Operating Characteristics analysis indicated that discriminability was higher in interactive compared to both photo and video lineups, demonstrating the benefit of actively exploring the lineup members’ faces. Signal-detection modeling suggested interactive lineups increase discriminability because they afford the witness the opportunity to view more diagnostic features such that the nondiagnostic features play a proportionally lesser role. These findings suggest that eyewitness errors can be reduced using interactive lineups because they create retrieval conditions that enable witnesses to actively explore faces and more effectively sample features.

[1] 120 Proceedings of the National Academies of Science (Oct. 10, 2023).