In my last post,[1] I praised Lee Mickus’s recent policy paper on amending Rule 702 for its persuasive force on the need for an amendment, as well as a source for helping lawyers anticipate common judicial dodges to a faithful application of the rule.[2] There are multiple dodges used by judicial dodgers, and it behooves litigants to recognize and anticipate them. In this post, and perhaps future ones, I elaborate upon the concerns that Mickus documents.
One prevalent judicial response to the Rule 702 motion is to kick the can and announce that the challenge to an expert witness’s methodological shenanigans can and should be addressed by crossexamination. This judicial response was, of course, the standard one before the 1993 Daubert decision, but Justice Blackmun’s opinion kept it alive in frequently quote dicta:
“Vigorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence.”[3]
Justice Blackmun, no doubt, believed he was offering a “helpful” observation here, but the reality is quite different. Traditionally, courts allowed qualified expert witnesses to opine with wild abandon, after showing that they had the very minimal qualifications required to do so in court. In the face of this traditional judicial lassitude, “[v]igorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof” were all a litigant could hope to accomplish in litigation. Furthermore, the litany of remedies for “shaky but admissible evidence” fails to help lower court judges and lawyers sort shaky but admissible evidence from shaky and inadmissible evidence.
Perhaps even more to the point, cases at common law “traditionally” did not involve multivariate logistic regression, structural equation models, propensity score weighting, and the like. Juries did just fine on whether Farmer Brown had exercised due care when he ran over his neighbor’s cow with his tractor, or even when a physician opined that a child was born 350 days after the putative father’s death was sired by the testator and entitled to inherit from “dad.”
Mickus is correct that a trial judge’s comment that the loser of a Rule 702 motion is free to cross-examine is often a dodge, an evasion, or an outright failure to engage with the intricacies of a complex methodological challenge.[4] Stating that the “traditional and appropriate means of attacking shaky but admissible evidence” remain available is a truism, and might be offered as judicial balm to the motion loser, but the availability of such means is hardly an explanation or justification for denying the Rule 702 motion. Furthermore, Justice Blackmun’s observation about traditional means was looking back at an era when in most state and federal court, a person found to be minimally qualified, could pretty much say anything regardless of scientific validity. That was the tradition that stood in active need of reform when Daubert was decided in 1993.
Mickus is also certainly correct that the whole point of judicial gatekeeping is that the presentation of vive voce testimony before juries is not an effective method for revealing shaky, inadmissible opinion testimony. A few courts have acknowledged that cross-examination in front of a jury is not an appropriate justification for admitting methodologically infirm expert witness opinion testimony. In the words of Judge Jed Rakoff, who served on the President’s Council of Advisors on Science and Technology,[5] addressed the limited ability of cross-examination in the context of forensic evidence:
“Although effective cross-examination may mitigate some of these dangers, the explicit premise of Daubert and Kumho Tire is that, when it comes to expert testimony, cross-examination is inherently handicapped by the jury’s own lack of background knowledge, so that the Court must play a greater role, not only in excluding unreliable testimony, but also in alerting the jury to the limitations of what is presented.”[6]
Judge Rakoff’s point is by no means limited to forensic evidence, and it has been acknowledged more generally by Professor Daniel Capra, the Reporter to the Advisory Committee on Evidence Rules:
“the key to Daubert is that cross-examination alone is ineffective in revealing nuanced defects in expert opinion testimony and that the trial judge must act as a gatekeeper to ensure that unreliable opinions don’t get to the jury in the first place.”[7]
Juries do not arrive at the court house knowledgeable about statistical and scientific methods; nor are they prepared to spend weeks going over studies to assess their quality, and whether an expert witness engaged in cherry picking, misapplying methodologies, or insufficient investigation.[8] In discussing the problem of expert witnesses’ overstating the strength of their opinions, beyond what is supported by evidence, the Reporter stressed the limits and ineffectiveness of remedial adversarial cross-examination:
“Perhaps another way to think about cross-examination as a remedy is to compare the overstatement issue to the issues of sufficiency of basis, reliability of methodology, and reliable application of that methodology. As we know, those three factors must be shown by a preponderance of the evidence. The whole point of Rule 702 — and the Daubert-Rule 104(a) gatekeeping function — is that these issues cannot be left to cross-examination. The underpinning of Daubert is that an expert’s opinion could be unreliable and the jury could not figure that out, even given cross-examination and argument, because the jurors are deferent to a qualified expert (i.e., the white lab coat effect). The premise is that cross-examination cannot undo the damage that has been done by the expert who has power over the jury. This is because, for the very reason that an expert is needed (because lay jurors need assistance) the jury may well be unable to figure out whether the expert is providing real information or junk. The real question, then, is whether the dangers of overstatement are any different from the dangers of insufficient basis, unreliability of methodology, and unreliable application. Why would cross-examination be insufficient for the latter yet sufficient for the former?
It is hard to see any difference between the risk of overstatement and the other risks that are regulated by Rule 702. When an expert says that they are certain of a result — when they cannot be — how is that easier for the jury to figure out than if an expert says something like ‘I relied on four scientifically valid studies concluding that PCB’s cause small lung cancer’. When an expert says he employed a ‘scientific methodology’ when that is not so, how is that different from an expert saying “I employed a reliable methodology” when that is not so?”[9]
The Reporter’s example of PCBs and small lung cancer was an obvious reference to the Joiner case, in which the Supreme Court held that the trial judge had properly excluded causation opinions. The Reporter’s point goes directly to the cross-examination excuse for not shirking the gatekeeping function. In Joiner, the Court held that gatekeeping was necessitated when cross-examination was insufficient in the face of an analytical gap between methodology and conclusion.[10] Indeed, such gaps are or should be present in most well-conceived Rule 702 challenges.
The problem is not only that juries defer to expert witnesses. Juries lack the competence to assess scientific validity. Although many judges are lacking in such competence, at least litigants can expect them to read the Reference Manual on Scientific Evidence before they read the parties’ briefs and the expert witnesses’ reports. If the trial judge’s opinion evidences ignorance of the Manual, then at least there is the possibility of an appeal. It will be a strange day in a stranger world, when a jury summons arrives in the mail with a copy of the Manual!
The rules of evidence permit expert witnesses to rely upon inadmissible evidence, at least when experts in their field would do so reasonably. To decide whether the reliance is reasonable requires the decision maker go outside the “proofs” that would typically be offered at trial. Furthermore, the decision maker – gatekeeper – will have to read the relied-upon study and data to evaluate the reasonableness of the reliance. In a jury trial, the actual studies relied upon are rarely admissible, and so the jury almost never has the opportunity to read them to make its own determination of reasonableness of reliance, or of whether the study and its data really support what the expert witness draws from it.
Of course, juries do not have to write opinions about their findings. They need neither explain nor justify their verdicts, once the trial court has deemed that there is the minimally sufficient evidence to support a verdict. Juries, with whatever help cross-examination provides, in the absence of gatekeeping, cannot deliver anything approaching scientific due process of law.
Despite Supreme Court holdings, a substantially revised and amended Rule 702, and clear direction from the Advisory Committee, some lower courts have actively resisted enforcing the requirements of Rule. 702 Part of this resistance consists in pushing the assessment of the reliability of the data and assumptions used in applying a given methodology out of the gatekeeping column and into the jury’s column. Despite the clear language of Rule 702, and the Advisory Committee Note,[11] some Circuits of the Court of Appeals have declared that assessing the reliability of assumptions and data is not judges’ work (outside of a bench trial).[12]
As Seinfeld has taught us, rules are like reservations. It is not enough to make the rules, you have to keep and follow them. Indeed, following the rule is really the important part.[13] Although an amended Rule 702 might include a provision that “we really mean this,” perhaps it is worth a stop at the Supreme Court first to put down the resistance.
[1] “Should Federal Rule of Evidence 702 Be Amended?” (May 8, 2020).
[2] Lee Mickus, “Gatekeeping Reorientation: Amend Rule 702 to Correct Judicial Misunderstanding about Expert Evidence,” Washington Legal Foundation Critical Legal Issues Working Paper No. 217 (May 2020).
[3] Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 596 (1993).
[4] See, e.g., AmGuard Ins. Co. v. Lone Star Legal Aid, No. CV H-18-2139, 2020 WL 60247, at *8 (S.D. Tex. Jan. 6, 2020) (“[O]bjections [that the expert could not link her experienced-based methodology to her conclusions] are better left for cross examination, not a basis for exclusion.”); Powell v. Schindler Elevator Corp., No. 3:14cv579 (WIG), 2015 WL 7720460, at *2 (D. Conn. Nov. 30, 2015) (“To the extent Defendant argues that Mr. McPartland’s conclusions are unreliable, it may attack his report through cross examination.”); Wright v. Stern, 450 F. Supp. 2d 335, 359–60 (S.D.N.Y. 2006) (“In a close case, a court should permit the testimony to be presented at trial, where it can be tested by cross-examination and measured against the other evidence in the case.”) (internal citation omitted). See also Adams v. Toyota Motor Corp., 867 F.3d 903, 916 (8th Cir. 2017) (affirming admission of expert testimony, reiterating the flexibility of the Daubert inquiry and emphasizing that defendant’s concerns could all be addressed with “[v]igorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof”); Liquid Dynamics Corp. v. Vaughan Corp., 449 F.3d 1209, 1221 (Fed. Cir. 2006) (“The identification of such flaws in generally reliable scientific evidence is precisely the role of cross-examination.” (internal citation omitted)); Carmichael v. Verso Paper, LLC, 679 F. Supp. 2d 109, 119 (D. Me. 2010) (“[W]hen the adequacy of the foundation for the expert testimony is at issue, the law favors vigorous cross-examination over exclusion.”); Crawford v. Franklin Credit Mgt. Corp., 08-CV-6293 (KMW), 2015 WL 13703301, at *6 (S.D.N.Y. Jan. 22, 2015) (“In light of the ‘presumption of admissibility of evidence,’ that opportunity [for cross-examination] is sufficient to ensure that the jury receives testimony that is both relevant and reliable.”) (internal citation omitted).
Even the most explicitly methodological challenges are transmuted into cross-examination issues by refusnik courts. For instance, cherry picking is reduced to a credibility issue for the jury and not germane to the court’s Rule 702 determination. In re Chantix Prods. Liab. Litig., 889 F. Supp. 2d 1272, 1288 (N.D. Ala. 2012) (finding that an expert witness’s deliberate decision not to rely upon clinical trial data merely “is a matter for cross-examination, not exclusion under Daubert”); In re Urethane Antitrust Litig., 2012 WL 6681783, at *3 (D.Kan.) (“The extent to which [an expert] considered the entirety of the evidence in the case is a matter for cross-examination.”); Bouchard v. Am. Home Prods. Corp., 2002 WL 32597992, at *7 (N.D. Ohio) (“If the plaintiff believes that the expert ignored evidence that would have required him to substantially change his opinion, that is a fit subject for cross-examination.”). Similarly, courts have by ipse dixit made flawed application of what a standard methodological into merely a credibility issue to be explore by cross-examination rather than by judicial gatekeeping. United States v. Adam Bros. Farming, 2005 WL 5957827, at *5 (C.D. Cal. 2005) (“Defendants’ objections are to the accuracy of the expert’s application of the methodology, not the methodology itself, and as such are properly reserved for cross-examination.”); Oshana v. Coca-Cola Co., 2005 WL 1661999, at *4 (N.D. Ill.) (“Challenges addressing flaws in an expert’s application of reliable methodology may be raised on cross-examination.”).
[5] President’s Council of Advisors on Science and Technology, Report to the President on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (Sept. 2016).
[6] United States v. Glynn, 578 F. Supp. 2d 567, 574 (S.D.N.Y. 2008) (Rakoff, J.)
[7] Daniel J. Capra, Reporter, Advisory Comm. on Evidence Rules, Minutes of Meeting at 23 (May 3, 2019) (comments of the Reporter).
[8] Daniel J. Capra, Reporter’s Memorandum re Forensic Evidence, Daubert and Rule 702 at 50 (April 1, 2018) (identifying issues such as insufficient investigation, cherry-picking data, or misapplying standard methodologies, as examples of a “white lab coat” problem resulting from juries’ inability to evaluate expert witnesses’ factual bases, methodologies, and applications of methods).
[9] Daniel J. Capra, Reporter, Advisory Comm. on Evidence Rules, Minutes of Meeting at 10-11 (Oct. 1, 2019) (comments of the Reporter on possible amendment of Rule 702) (internal citation to Joiner omitted).
[10] Id. at 11 n.5.
[11] See In re Paoli RR Yard PCB Litig., 35 F.3d 717, 745 (3d Cir. 1994) (calling for a close, careful analysis of the application of a proper methodology to every step in the case; “any step that renders the analysis unreliable renders the expert’s testimony inadmissible whether the step completely changes a reliable methodology or merely misapplies that methodology”).
[12] See, e.g., City of Pomona v. SQM North Am. Corp., 750 F.3d 1036, 1047 (9th Cir. 2014) (rejecting the Paoli any-step approach without careful analysis of the statute, the advisory committee note, or Supreme Court decisions); Manpower, Inc. v. Ins. Co. of Pa., 732 F.3d 796, 808 (7th Cir. 2013) (“[t]he reliability of data and assumptions used in applying a methodology is tested by the adversarial process and determined by the jury; the court’s role is generally limited to assessing the reliability of the methodology – the framework – of the expert’s analysis”); Bonner v. ISP Techs., Inc., 259 F.3d 924, 929 (8th Cir. 2001) (“the factual basis of an expert opinion goes to the credibility of the testimony, not the admissibility, and it is up to the opposing party to examine the factual basis for the opinion in cross-examination”).
[13] Despite the clarity of the revised Rule 702, and the intent to synthesize Daubert, Joiner, Kumho Tire, and Weisgram, some courts have insisted that nothing changed with the amended rule. See, e.g., Pappas v. Sony Elec., Inc., 136 F. Supp. 2d 413, 420 & n.11 (W.D. Pa. 2000) (opining that Rule 702 as amended did not change the application of Daubert within the Third Circuit) (“The Committee Notes to the amended Rule 702 cite and discuss several Court of Appeals decisions that have properly applied Daubert and its progeny. Among these decisions are numerous cases from the Third Circuit. See Committee Note to 2000 Amendments to Fed. R.Evid. 702. Accordingly, I conclude that amended Rule 702 does not effect a change in the application of Daubert in the Third Circuit.”). Of course, if nothing changed, then the courts that take this position should be able to square their decisions with text of Rule 702, as amended in 2000.
