In In re Zurn Pex Plumbing Prods. Liab. Litig., 644 F.3d 604 (8th Cir. 2011), the United States Court of Appeals for the Eighth Circuit rejected the defendant’s argument that a “full and conclusive” Rule 702 gatekeeping procedure was required before a trial court could certify a class action under the Federal Rules. The Circuit remarked that “[t]he main purpose of Daubert exclusion is to protect juries from being swayed by dubious scientific testimony,” an interest “not implicated at the class certification stage where the judge is the decision maker.” Id. at 613.
Surely, one important purpose of Rule 702 is to protect juries against dubious scientific testimony, but judges are not universally less susceptible to dubious testimony. There are many examples of judges being misled by fallacious scientific evidence, especially when tendentiously presented by advocates in court. No jury need be present for dubious science testimony + “zealous” advocacy to combine to create major errors and injustice. See, e.g., Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D. Ga. 1985)(rendering verdict for plaintiffs after bench trial), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986); Hans Zeisel & David Kaye, Prove It With Figures: Empirical Methods in Law and Litigation § 6.5 n.3, at 271 (1997) (characterizing Wells as “notorious,” and noting that the case became a “lightning rod for the legal system’s ability to handle expert evidence.”). Clearly Rule 702 does not exist only to protect juries.
Nemo iudex in causa sua! Perhaps others should judge the competence of judges’ efforts at evaluating scientific evidence. At the very least, within the institutional framework of our rules of civil procedure and evidence, Rule 702 creates a requirement of structured inquiry into expert opinion testimony before the court. That gatekeeping inquiry, and its requirement of a finding, subject to later appellate review and to public and professional scrutiny, are crucial to the rendering of intellectual due process in cases that involve scientific and technical issues. The Eighth Circuit was unduly narrow in its statement of the policy bases for Rule 702, and their applicability to class certification.
The case of Obrey v. Johnson, 400 F.3d 691 (9th Cir. 2005) provides another cautionary tale about the inadequacies of judges in the evaluation of scientific and statistical evidence. The plaintiff, Mr. Obrey, sued the Navy on a claim of race discrimination in promoting managers at the Pearl Harbor Naval Shipyard. The district court refused plaintiff’s motion to admit the testimony of a statistician, Mr. James Dannemiller, President of the SMS Research & Marketing Services, Inc. The district court also excluded much of plaintiff’s anecdotal evidence, and entered summary judgment. Id. at 691 – 93.
On appeal, Obrey claimed that Dannemiller’s report showed “a correlation between race and promotion.” Id. at 693. This vague claim seemed good enough for the Ninth Circuit, which reversed the district court’s grant of summary judgment and remanded for trial.
The Ninth Circuit’s opinion does not tell us what sort of correlation was supposedly shown by Mr. Dannemiller. Was it Pearson’s r? Or Jaspen’s multi-serial coefficient? Spearman’s ρ? Perhaps Kendall’s τ? Maybe the appellate court was using correlation loosely, and Mr. Dannemiller had conducted some other sort of statistical analysis. The district court’s opinion is not published and is not available on Westlaw. It is all a mystery. More process is due the litigants and the public.
Even more distressing than the uncertainty as to the nature of the correlation is that the Ninth Circuit does not tell us what the correlation “effect size” was, or whether the correlation was statistically significant. If the Circuit did not follow strict hypothesis testing, perhaps it might have told us the extent of random error in the so-called correlation. The Circuit did not provide any information about the extent or the precision of the claim of a “correlation”; nor did the Circuit assess the potential for bias or confounding in Mr. Dannemiller’s analysis.
Indeed, the Ninth Circuit seemed to suggest that Mr. Dannemiller never even showed a correlation; rather the court described Mr. Dannemiller as having opined that there was “no statistical evidence in these data that the selection process for GS-13 through GS-15 positions between 1999 and 2002 was unbiased with respect to race.” Id. at 694. Reading between the lines, it seems that the statistical evidence was simply inconclusive, and Mr. Dannemiller surreptitiously shifted the burden of proof and offered an opinion that the Navy had not ruled out bias. The burden, of course, was on Mr. Obrey to establish a prima facie case, but the appellate court glossed over this fatal gap in plaintiff’s evidence.
On appeal, the Navy pressed its objections to the relevance and reliability of Mr. Dannemiller’s opinions. Brief of the Navy, 2004 WL 1080083, at *1 (April 7, 2004). There seemed to be no dispute that Mr. Dannemiller’s “study” was based entirely upon “statistical disparities,” which failed to take into account education, experience, and training. Mr. Dannemiller appeared to have simplistically compared race make up of the promoted workers, ignoring the Navy’s showing of the relevancy of education, experience, and training. Id. at *13, 18.
The Ninth Circuit not only ignored the facts of the case, it ignored its own precedents. See Obrey v. Johnson, 400 F.3d at 696 (citing and quoting from Coleman v. Quaker Oats Co., 232 F.3d 1271, 1283 (9th Cir. 2000) (“Because [the statistics] failed to account for many factors pertinent to [the plaintiff], we conclude that the statistics are not enough to take this case to trial.”). The court, in Obrey, made no effort to distinguish its treatment of the parties in Coleman, or to justify its decision as to why the unspecified, unquantified, mysterious statistical analysis of Mr. Dannemiller sufficed under Rule 702. The Circuit cryptically announced that “Obrey’s evidence was not rendered irrelevant under Rule 402 simply because it failed to account for the relative qualifications of the applicant pool.” Obrey, 400 F.3d at 695. Citing pre-Daubert decisions for the most part (such as Bazemore), the Ninth Circuit persuaded itself that Rule 702 requires nothing more than simple relevancy. Had the Circuit taken even a cursory look at Bazemore, it would have seen that the case involved a much more involved multiple regression than whatever statistical analysis Mr. Dannemiller propounded. And the Ninth Circuit would have seen that even the Bazemore decision acknowledged that there may be
“some regressions so incomplete as to be inadmissible as irrelevant… .”
478 U.S. 385, 400 n.10 (1986). It is difficult to imagine a discrimination claim analysis more incomplete than one that did not address education, training, and experience.
Sadly, neither the Navy’s nor Mr. Obrey’s brief, 2004 WL 545873 (Feb. 4, 2004) provided any discussion of the nature, quality, findings, or limits of Mr. Dannemiller’s statistical analysis. The Navy’s brief referred to Mr. Dannemiller as a “purported” expert. His resume, available online, shows that Mr. Dannemiller studied history as an undergraduate, and has a master’s degree in sociology. He is the president of SMS Research, a consulting company.
The taxpayers deserved better advocacy from the Department of Justice, and greater attention to statistical methodology from its appellate judges. See ATA Airlines, Inc. v. Federal Exp. Corp., 665 F.3d 882, 888-96 (2011) (Posner, J.) (calling for lawyers and judges to do better in understanding and explaining, in plain English, the statistical analyses that are essential to their cases). Judges at level need to pay greater attention to the precepts of Rule 702, even when there is no jury around to be snuckered.
